%% CS294A/CS294W Self-taught Learning Exercise%  Instructions%  ------------% %  This file contains code that helps you get started on the%  self-taught learning. You will need to complete code in feedForwardAutoencoder.m%  You will also need to have implemented sparseAutoencoderCost.m and %  softmaxCost.m from previous exercises.%%% ======================================================================%  STEP 0: Here we provide the relevant parameters values that will%  allow your sparse autoencoder to get good filters; you do not need to %  change the parameters below.inputSize  = 28 * 28;numLabels  = 5;hiddenSize = 200;sparsityParam = 0.1; % desired average activation of the hidden units.                     % (This was denoted by the Greek alphabet rho, which looks like a lower-case "p",                     %  in the lecture notes). lambda = 3e-3;       % weight decay parameter       beta = 3;            % weight of sparsity penalty term   maxIter = 400;%% ======================================================================%  STEP 1: Load data from the MNIST database%%  This loads our training and test data from the MNIST database files.%  We have sorted the data for you in this so that you will not have to%  change it.% Load MNIST database filesmnistData   = loadMNISTImages(‘mnist/train-images-idx3-ubyte‘);mnistLabels = loadMNISTLabels(‘mnist/train-labels-idx1-ubyte‘);% Set Unlabeled Set (All Images)% Simulate a Labeled and Unlabeled setlabeledSet   = find(mnistLabels >= 0 & mnistLabels <= 4);unlabeledSet = find(mnistLabels >= 5);numTrain = round(numel(labeledSet)/2);trainSet = labeledSet(1:numTrain);testSet  = labeledSet(numTrain+1:end);unlabeledData = mnistData(:, unlabeledSet);trainData   = mnistData(:, trainSet);trainLabels = mnistLabels(trainSet)‘ + 1; % Shift Labels to the Range 1-5testData   = mnistData(:, testSet);testLabels = mnistLabels(testSet)‘ + 1;   % Shift Labels to the Range 1-5% Output Some Statisticsfprintf(‘# examples in unlabeled set: %d\n‘, size(unlabeledData, 2));fprintf(‘# examples in supervised training set: %d\n\n‘, size(trainData, 2));fprintf(‘# examples in supervised testing set: %d\n\n‘, size(testData, 2));%% ======================================================================%  STEP 2: Train the sparse autoencoder%  This trains the sparse autoencoder on the unlabeled training%  images. %  Randomly initialize the parameterstheta = initializeParameters(hiddenSize, inputSize);%% ----------------- YOUR CODE HERE ----------------------%  Find opttheta by running the sparse autoencoder on%  unlabeledTrainingImagesopttheta = theta; %  Use minFunc to minimize the functionaddpath minFunc/options.Method = ‘lbfgs‘; % Here, we use L-BFGS to optimize our cost                          % function. Generally, for minFunc to work, you                          % need a function pointer with two outputs: the                          % function value and the gradient. In our problem,                          % sparseAutoencoderCost.m satisfies this.options.maxIter = maxIter;% Maximum number of iterations of L-BFGS to run options.display = ‘on‘;[opttheta, cost] = minFunc( @(p) sparseAutoencoderCost(p, ...                                   inputSize, hiddenSize, ...                                   lambda, sparsityParam, ...                                   beta, trainData), ...                              opttheta, options);%% -----------------------------------------------------                          % Visualize weightsW1 = reshape(opttheta(1:hiddenSize * inputSize), hiddenSize, inputSize);display_network(W1‘);%%======================================================================%% STEP 3: Extract Features from the Supervised Dataset%  %  You need to complete the code in feedForwardAutoencoder.m so that the %  following command will extract features from the data.trainFeatures = feedForwardAutoencoder(opttheta, hiddenSize, inputSize, ...                                       trainData);testFeatures = feedForwardAutoencoder(opttheta, hiddenSize, inputSize, ...                                       testData);%%======================================================================%% STEP 4: Train the softmax classifiersoftmaxModel = struct;  %% ----------------- YOUR CODE HERE ----------------------%  Use softmaxTrain.m from the previous exercise to train a multi-class%  classifier. %  Use lambda = 1e-4 for the weight regularization for softmax% You need to compute softmaxModel using softmaxTrain on trainFeatures and% trainLabelslambda = 1e-4;options2.maxIter = maxIter;softmaxModel = softmaxTrain(hiddenSize, numLabels, lambda, ...                            trainFeatures, trainLabels, options2);%% -----------------------------------------------------%%======================================================================%% STEP 5: Testing %% ----------------- YOUR CODE HERE ----------------------% Compute Predictions on the test set (testFeatures) using softmaxPredict% and softmaxModel[pred] = softmaxPredict(softmaxModel, testFeatures);%% -----------------------------------------------------% Classification Scorefprintf(‘Test Accuracy: %f%%\n‘, 100*mean(pred(:) == testLabels(:)));% (note that we shift the labels by 1, so that digit 0 now corresponds to%  label 1)%% Accuracy is the proportion of correctly classified images% The results for our implementation was:%% Accuracy: 98.3%%% 

function [activation] = feedForwardAutoencoder(theta, hiddenSize, visibleSize, data)% theta: trained weights from the autoencoder% visibleSize: the number of input units (probably 64) % hiddenSize: the number of hidden units (probably 25) % data: Our matrix containing the training data as columns.  So, data(:,i) is the i-th training example.   % We first convert theta to the (W1, W2, b1, b2) matrix/vector format, so that this % follows the notation convention of the lecture notes. W1 = reshape(theta(1:hiddenSize*visibleSize), hiddenSize, visibleSize);b1 = theta(2*hiddenSize*visibleSize+1:2*hiddenSize*visibleSize+hiddenSize);%% ---------- YOUR CODE HERE --------------------------------------%  Instructions: Compute the activation of the hidden layer for the Sparse Autoencoder.activation = sigmoid(W1 * data + repmat(b1,1,size(data,2)));%-------------------------------------------------------------------end%-------------------------------------------------------------------% Here‘s an implementation of the sigmoid function, which you may find useful% in your computation of the costs and the gradients.  This inputs a (row or% column) vector (say (z1, z2, z3)) and returns (f(z1), f(z2), f(z3)). function sigm = sigmoid(x)    sigm = 1 ./ (1 + exp(-x));end