1 / 6

Datamining & Neural Networks

Datamining & Neural Networks . Session 3 PCA & ARD. Regularization on cost-function : E D + E W e.g E W =1/2*||w|| 2 three levels of inference Infer model parameters w ; ,  fixed Infer ,  ; given the model params comparison of different models

lucky
Download Presentation

Datamining & Neural Networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Datamining & Neural Networks Session 3 PCA & ARD

  2. Regularization on cost-function : ED + EW • e.g EW=1/2*||w||2 • three levels of inference • Infer model parameters w ; ,  fixed • Infer ,  ; given the model params • comparison of different models • Evidence-based framework of MacKay for estimation of hyperparameters : interpretation of •  as 1/2W •  as 1/2D • approx. of evidence factor yields update formula’s for hyperparameters P(|D)~P(D|) * P()

  3. ARD • one hyperparam per input of your NN • Goal: detection of most relevant inputs in the Bayesian framework 1 2 N N 3 Costfunction = ED + 1(wi12)+2(wi22)+…

  4. ARD in practice • Define NN model • Optimize weights • Optimize ,  , -- given weights • Cycle back to 2.

  5. Exercise 2 • Demev1 (Evidence-based framework) • Demard (ARD) • Ionosphere dataset: 33 inputs, appr. 350 datapoints. • make classification NN (cfr. also session 2, ex. 3) • perform ARD, rank inputs • select subset of inputs based on ’s • make new NN with this subset of inputs • compare performance with first NN

More Related