440 likes | 466 Views
Lecture 7-II Nonlinear Separation. Data Classification Linear Separation Nonlinear Separation MLP learning for nonlinear separation. Data with labels. Example. d=2 x[t] : data coordinate y[t] : data color. Example. Iris data d=4
E N D
Lecture 7-II NonlinearSeparation • Data Classification • Linear Separation • Nonlinear Separation • MLP learning for nonlinear separation 數值方法2008, Applied Mathematics NDHU
Data with labels 數值方法2008, Applied Mathematics NDHU
Example • d=2 • x[t] : data coordinate • y[t] : data color 數值方法2008, Applied Mathematics NDHU
Example Iris data d=4 y[t] {Iris-setosa, Iris-versicolor, Iris-virginica} x[t]=(x1[t], x2[t], x3[t], x4[t]) iris.txt UCI Machine Learning Repository x1: sepal length in cm x2: sepal width in cm x3: petal length in cm x4: petal width in cm 數值方法2008, Applied Mathematics NDHU
Example • Wine data • Breast Cancer Wisconsin (Diagnostic) 數值方法2008, Applied Mathematics NDHU
2D labeled data Linear Separation Linear Separation 數值方法2008, Applied Mathematics NDHU
Linear Separation 數值方法2008, Applied Mathematics NDHU
Over-determined linear system 數值方法2008, Applied Mathematics NDHU
Least square method 數值方法2008, Applied Mathematics NDHU
Least square method • Let X denote data matrix of size Nxd • A=[X ones(N,1)] • pinv(A)*y 數值方法2008, Applied Mathematics NDHU
Toolkit for data separation Toolkit for data separation 數值方法2008, Applied Mathematics NDHU
Press New to start • Press Pendata to paint blue points • Press OK to get data • Press Pendata to paint red points • Goto step 2 • or press Linear Separation • Press SAVE to store paired data • Press LOAD to get paired data 數值方法2008, Applied Mathematics NDHU
Linearly separable 數值方法2008, Applied Mathematics NDHU
Linearly separable 數值方法2008, Applied Mathematics NDHU
Coding • a=finda(x,y) • Given x and y, find vector a to minimize the mean square error • yhat=x2y(x,a) • Map x to y for given a 數值方法2008, Applied Mathematics NDHU
finda.m 數值方法2008, Applied Mathematics NDHU
x2y.m 數值方法2008, Applied Mathematics NDHU
SeparableTest.m Calling finda and x2y 數值方法2008, Applied Mathematics NDHU
Linearly non-separable • Non-separable data by linear separation 數值方法2008, Applied Mathematics NDHU
Non-linear separation 數值方法2008, Applied Mathematics NDHU
plot2d.m 數值方法2008, Applied Mathematics NDHU
Linear projection 數值方法2008, Applied Mathematics NDHU
Two linear projections • Add two linear projections 數值方法2008, Applied Mathematics NDHU
MLP Networks b1 tanh r1 b2 1 a1 tanh r2 a2 bM x ……. rM aM tanh rM+1 1 數值方法2008, Applied Mathematics NDHU
Install NNSYSID • Install • Download • Set path to recruit the directory where NNSYSID.zip is extracted The NNSYSID Toolbox findabr.m x2y_MLP.m (Levenberg Marquardt method) 數值方法2008, Applied Mathematics NDHU
fa2d.m Calling findabr Calling x2y_MLP 數值方法2008, Applied Mathematics NDHU
fa2d MSE for training data 0.008656 ME for training data 0.065931>> 數值方法2008, Applied Mathematics NDHU
fa2d 數值方法2008, Applied Mathematics NDHU
SeparableTest.m 數值方法2008, Applied Mathematics NDHU
Nonlinear separation by NNSYSIS • Accomplish ex.m • There are three lines in ex.m • They are employed to replace lines 353-355 • Please refer fa2d.m to accomplish these three lines • Calling findabr.m • Calling x2y_MLP 數值方法2008, Applied Mathematics NDHU
Nonlinear Separation Linearly separable data 數值方法2008, Applied Mathematics NDHU
M=20 數值方法2008, Applied Mathematics NDHU
Sunspot time series sunpot data load sunspot_train.mat; n=length(Y); plot(1712:1712+n-1,Y); 數值方法2008, Applied Mathematics NDHU
Paired data x=[]; len=3; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end figure; plot3(x(2,:),x(1,:),y,'.');hold on; 數值方法2008, Applied Mathematics NDHU
Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2:1712+2+length(y)-1,y,'r') MSE for training data 0.005029 ME for training data 0.052191K 數值方法2008, Applied Mathematics NDHU
Prediction 數值方法2008, Applied Mathematics NDHU
Mesh x1=0:0.02:max(max(abs(x))); x2=x1; for i=1:length(x1) x_test = [x1(i)*ones(1,length(x2));x2]; y=eval_MLP2(x_test,r,a,b,M); C(i,:)=y; end mesh(x1,x2,C); 數值方法2008, Applied Mathematics NDHU
Sunspot Source codes 數值方法2008, Applied Mathematics NDHU
Paired data x=[]; len=5; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end 數值方法2008, Applied Mathematics NDHU
Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2:1712+2+length(y)-1,y,'r') ??? 數值方法2008, Applied Mathematics NDHU
Example Repeat numerical experiments of slide #42 separately using Use a table to summarize your numerical results 數值方法2008, Applied Mathematics NDHU