1 / 44

Lecture 7-II Nonlinear Separation

Lecture 7-II Nonlinear Separation. Data Classification Linear Separation Nonlinear Separation MLP learning for nonlinear separation. Data with labels. Example. d=2 x[t] : data coordinate y[t] : data color. Example. Iris data d=4

rodriqueza
Download Presentation

Lecture 7-II Nonlinear Separation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 7-II NonlinearSeparation • Data Classification • Linear Separation • Nonlinear Separation • MLP learning for nonlinear separation 數值方法2008, Applied Mathematics NDHU

  2. Data with labels 數值方法2008, Applied Mathematics NDHU

  3. Example • d=2 • x[t] : data coordinate • y[t] : data color 數值方法2008, Applied Mathematics NDHU

  4. Example Iris data d=4 y[t] {Iris-setosa, Iris-versicolor, Iris-virginica} x[t]=(x1[t], x2[t], x3[t], x4[t]) iris.txt UCI Machine Learning Repository x1: sepal length in cm x2: sepal width in cm x3: petal length in cm x4: petal width in cm 數值方法2008, Applied Mathematics NDHU

  5. Example • Wine data • Breast Cancer Wisconsin (Diagnostic) 數值方法2008, Applied Mathematics NDHU

  6. 2D labeled data Linear Separation Linear Separation 數值方法2008, Applied Mathematics NDHU

  7. Linear Separation 數值方法2008, Applied Mathematics NDHU

  8. Over-determined linear system 數值方法2008, Applied Mathematics NDHU

  9. Least square method 數值方法2008, Applied Mathematics NDHU

  10. Least square method • Let X denote data matrix of size Nxd • A=[X ones(N,1)] • pinv(A)*y 數值方法2008, Applied Mathematics NDHU

  11. Toolkit for data separation Toolkit for data separation 數值方法2008, Applied Mathematics NDHU

  12. Press New to start • Press Pendata to paint blue points • Press OK to get data • Press Pendata to paint red points • Goto step 2 • or press Linear Separation • Press SAVE to store paired data • Press LOAD to get paired data 數值方法2008, Applied Mathematics NDHU

  13. Linearly separable 數值方法2008, Applied Mathematics NDHU

  14. Linearly separable 數值方法2008, Applied Mathematics NDHU

  15. Coding • a=finda(x,y) • Given x and y, find vector a to minimize the mean square error • yhat=x2y(x,a) • Map x to y for given a 數值方法2008, Applied Mathematics NDHU

  16. finda.m 數值方法2008, Applied Mathematics NDHU

  17. x2y.m 數值方法2008, Applied Mathematics NDHU

  18. SeparableTest.m Calling finda and x2y 數值方法2008, Applied Mathematics NDHU

  19. 數值方法2008, Applied Mathematics NDHU

  20. Linearly non-separable • Non-separable data by linear separation 數值方法2008, Applied Mathematics NDHU

  21. Non-linear separation 數值方法2008, Applied Mathematics NDHU

  22. plot2d.m 數值方法2008, Applied Mathematics NDHU

  23. Linear projection 數值方法2008, Applied Mathematics NDHU

  24. Two linear projections • Add two linear projections 數值方法2008, Applied Mathematics NDHU

  25. 數值方法2008, Applied Mathematics NDHU

  26. 數值方法2008, Applied Mathematics NDHU

  27. MLP Networks b1 tanh r1 b2 1 a1 tanh r2 a2 bM x ……. rM aM tanh rM+1 1 數值方法2008, Applied Mathematics NDHU

  28. Install NNSYSID • Install • Download • Set path to recruit the directory where NNSYSID.zip is extracted The NNSYSID Toolbox findabr.m x2y_MLP.m (Levenberg Marquardt method) 數值方法2008, Applied Mathematics NDHU

  29. fa2d.m Calling findabr Calling x2y_MLP 數值方法2008, Applied Mathematics NDHU

  30. fa2d MSE for training data 0.008656 ME for training data 0.065931>> 數值方法2008, Applied Mathematics NDHU

  31. fa2d 數值方法2008, Applied Mathematics NDHU

  32. SeparableTest.m 數值方法2008, Applied Mathematics NDHU

  33. Nonlinear separation by NNSYSIS • Accomplish ex.m • There are three lines in ex.m • They are employed to replace lines 353-355 • Please refer fa2d.m to accomplish these three lines • Calling findabr.m • Calling x2y_MLP 數值方法2008, Applied Mathematics NDHU

  34. Nonlinear Separation Linearly separable data 數值方法2008, Applied Mathematics NDHU

  35. M=20 數值方法2008, Applied Mathematics NDHU

  36. Sunspot time series sunpot data load sunspot_train.mat; n=length(Y); plot(1712:1712+n-1,Y); 數值方法2008, Applied Mathematics NDHU

  37. Paired data x=[]; len=3; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end figure; plot3(x(2,:),x(1,:),y,'.');hold on; 數值方法2008, Applied Mathematics NDHU

  38. Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2:1712+2+length(y)-1,y,'r') MSE for training data 0.005029 ME for training data 0.052191K 數值方法2008, Applied Mathematics NDHU

  39. Prediction 數值方法2008, Applied Mathematics NDHU

  40. Mesh x1=0:0.02:max(max(abs(x))); x2=x1; for i=1:length(x1) x_test = [x1(i)*ones(1,length(x2));x2]; y=eval_MLP2(x_test,r,a,b,M); C(i,:)=y; end mesh(x1,x2,C); 數值方法2008, Applied Mathematics NDHU

  41. Sunspot Source codes 數值方法2008, Applied Mathematics NDHU

  42. Paired data x=[]; len=5; for i=1:n-len p=Y(i:i+len-1)'; x=[x p]; y(i)=Y(i+len); end 數值方法2008, Applied Mathematics NDHU

  43. Learning and prediction M=input('keyin the number of hidden units:'); [a,b,r]=learn_MLP(x',y',M); y=eval_MLP2(x,r,a,b,M); hold on; plot(1712+2:1712+2+length(y)-1,y,'r') ??? 數值方法2008, Applied Mathematics NDHU

  44. Example Repeat numerical experiments of slide #42 separately using Use a table to summarize your numerical results 數值方法2008, Applied Mathematics NDHU

More Related