1 / 24

Applying Neural Networks Michael J. Watts mike.watts.nz

This article provides an overview of applying neural networks in various problem scenarios. It covers topics such as data preparation, choosing the appropriate type of neural network, training techniques, and the importance of careful data analysis. The article emphasizes the trial and error process involved in optimizing neural networks for each specific problem.

schapman
Download Presentation

Applying Neural Networks Michael J. Watts mike.watts.nz

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Applying Neural Networks Michael J. Watts http://mike.watts.net.nz

  2. Lecture Outline When to use an ANN Preparing the data Apportioning data What kind of ANN to use Training ANN

  3. When to use an ANN When the rules of the problem aren’t known When there is an underlying function ANN can’t model random processes! ANN learn from existing data Careful preparation of the data is essential!

  4. Preparing the Data Need a numerical representation of inputs Beware when encoding discrete items! nominal vs. ordinal Binary coding should be -1,1 not 0,1 hyperplane geometry & small initial weights most published application papers ignore this, however

  5. Preparing the Data Identify essential variables data analysis Redundant variables complicate the learning process curse of dimensionality Need more training examples than inputs Remove / transform outliers

  6. Preparing the Data Normalisation not strictly necessary speeds training sigmoid, tanh functions again, either -1,1 or 0,1 set mean 0 STD 1 MATLAB MLP simulator normalises inputs automatically

  7. Preparing the Data Two kinds of problems classification function approximation / time series Classification has binary outputs Function approximation has continuous outputs

  8. Preparing the Data Two ways of encoding multiple classes One output per class sparse encoding e.g. 1 0 0, 0 1 0, 0 0 1 Combining outputs dense encoding e.g. 1 0, 0 1, 0 0

  9. Preparing the Data Some people use 0.1 & 0.9 instead of 0 & 1 Used to speed up training limits of sigmoid function This is a gimmick Cannot interpret outputs as probabilities Better training algorithm should be used

  10. Apportioning Data Splitting up the available data into multiple sets Two or three sets training testing validation Each set has a different purpose

  11. Apportioning Data Training data set is used to train the network Larger of the three sets Usually 50-75% of the data Should represent a wide range of output classes / values mustn’t omit any target classes

  12. Apportioning Data Testing data set is used to evaluate the performance of the trained network Network has not been trained on testing data Accuracy over testing set is an indicator of generalisation ability of the network

  13. Apportioning Data 25% of the total data if no validation set it used Otherwise, half of the remaining data Must also represent an adequate range of output classes / values

  14. Apportioning Data Validation data set is used as a final test Performance over this set is evaluated once Guards against optimising the network to the training set

  15. Apportioning Data Similar to over-training Validation set consists of all remaining data As before, must have a wide range of output classes / values

  16. Apportioning Data Can’t just split a data set into three to form training, testing and validation sets Statistical parameters of each set should be similar Distribution of target classes / values should be similar can’t have all of class A in the training set and all of class B in the testing set

  17. What Kind of ANN to use Perceptrons are useful for linearly separable problems MLP handle problems perceptrons cannot non-linearly separable Both use supervised learning require known outputs for each example

  18. What Kind of ANN to use Kohonen SOMs are good for classification Can’t handle function approximation very well Unsupervised learning no outputs required Find clusters in the data

  19. Training ANN Goal is to maximise generalisation accuracy Problems with over-training low training error high generalisation error Common practice is to train until a low training error, then evaluate generalisation on test set

  20. Training ANN Can lead to optimising on test set as well If the network is optimised for performance on the testing set, then the network is only good for that data set Use of validation data set is intended to counter this

  21. Training ANN Number of hidden neurons must be selected for MLPs Some heuristics exist for this most are not reliable To avoid over-training, number of connections should be less than the number of training examples this is itself a heuristic, though

  22. Training ANN Selection of parameters for backpropagation training Learning rate must be determined by trial and error Momentum less critical Number of epochs effects generalisation

  23. Training ANN Heuristics exist for each of these parameters Trial and error is the only reliable method Care must be taken in assessing generalisation selection of validation set

  24. Summary Applying ANN is something of a “black art” Each problem is different Careful preparation of data is critical Trial and error is often the only way to find an optimal network

More Related