1 / 21

Lecture 5 II Function Approximation

Lecture 5 II Function Approximation. Data driven function approximation MLP network Unconstrained optimization. Objective function. Unconstrained Optimization Non-polynomial, High dimensional function Unknowns: 3M+1. Unconstrained Optimization. Gradient method Gauss-Newton method

xenon
Download Presentation

Lecture 5 II Function Approximation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 5 II Function Approximation • Data driven function approximation • MLP network • Unconstrained optimization 數值方法

  2. Objective function • Unconstrained Optimization • Non-polynomial, • High dimensional function • Unknowns: 3M+1 數值方法

  3. Unconstrained Optimization • Gradientmethod • Gauss-Newton method • Levenberg-Marquardt method 數值方法

  4. Unconstrained Optimization 數值方法

  5. Iterative approach • Initialize with i=0 • Determine • Update network parameters • If halting condition holds, exit • otherwise i=i+1, go to step 2 數值方法

  6. Flow Chart i=0; guess ~ halting condition 數值方法

  7. Gradient method • Derive the derivative of ES with respect to • Substitute to 數值方法

  8. Install NNSYSID • Install • Download • Set path to recruit the directory where NNSYSID.zip is extracted The NNSYSID Toolbox learn_MLP By Levenberg Marquardt method 數值方法

  9. Data creation Get a function fstr=input('input a function: x.^2+cos(x) :','s'); fx=inline(fstr); range=2*pi; x=linspace(-range,range); y=fx(x);max_y=max(y); plot(x,y/max_y); Function plotting 數值方法

  10. Sampling N=input('keyin sample size:'); x=rand(1,N)*2*range-range; n=rand(1,N)*0.1-0.05; y=fx(x)/max_y+n; plot(x,y,'.'); 數值方法

  11. MLP learning M=5; [a,b,r]=learn_MLP(x',y',M); MLP learning x=linspace(-range,range); y=eval_MLP(x,r,a,b,M); plot(x,y,'r') MLP evaluation 數值方法

  12. Data Driven Function Approximation source code mean_square_error2.m eval_MLP.m 數值方法

  13. Example >> fa1d input a function: x.^2+cos(x) :3*x.^2+2*x+1 keyin sample size:300 keyin the number of hidden units:5 數值方法

  14. Example >> fa1d input a function: x.^2+cos(x) :cos(x)+sin(x) keyin sample size:300 keyin the number of hidden units: 數值方法

  15. High-dimensional function approximation Given Find a parametric function to minimize 數值方法

  16. Network b1 tanh r1 b2 1 a1 tanh r2 a2 x bM ……. rM aM tanh rM+1 1 數值方法

  17. Example Black box plot2d.m observations Create paired data sampling2.m y=cos(x1+x2) DDFA M=input('M:'); [a,b,r]=learn_MLP(x',y',M); 數值方法

  18. D-dimensional function approximation source codes eval_MLP2.m 數值方法

  19. Example >> fa2d input a 2D function: x1.^2+x2.^2+cos(x1) :x1.^2-x2.^2 keyin sample size:300 keyin the number of hidden units:20 數值方法

  20. Numerical results MSE for training data 0.008656 ME for training data 0.065931>> 數值方法

  21. Example 數值方法

More Related