1 / 25

Multilayer feed-forward artificial neural networks for Class-modeling

Multilayer feed-forward artificial neural networks for Class-modeling. F. Marini , A. Magrì, R. Bucci Dept. of Chemistry - University of Rome “La Sapienza”. The starting question….

roz
Download Presentation

Multilayer feed-forward artificial neural networks for Class-modeling

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Multilayer feed-forwardartificial neural networks for Class-modeling F. Marini, A. Magrì, R. Bucci Dept. of Chemistry - University of Rome “La Sapienza”

  2. The starting question…. Despite literature on NNs has increased significantly, no paper considers the possibility of performing class-modeling

  3. class modeling: what…. • Class modeling considers one class at a time • Any object can then belong or not to that specific class model • As a consequence, any object can be assigned to only one class, to more than one class or to no class at all classification class modeling

  4. less equivocal answer to the question: “are the analytical data compatible with the product being X as declared?” …..and why • Flexibility • Additional information: • sensitivity: fraction of samples from category X accepted by the model of category X • specificity: fraction of samples from category Y (or Z, W….) refused by the model of category X • No need to rebuild the existing models each time a new category is added.

  5. A first step forward • A particular kind of NN, after suitable modifications could be used for performing class-modeling (Anal. Chim. Acta, 544 (2005), 306) • Kohonen SOM • Addition of dummy random vectors to the training set • Computation of a suitable (non-parametric) probability distribution after mapping on the 2D Kohonen layer. • Definition of the category space based on this distribution

  6. In this communication… …The possibility of using a different type of neural network (multilayer feed-forward) to operate class-modeling is studied • How to? • Examples

  7. Just a few words about NN Polla ta deina kouden an- qropou deinoteron pelei. Sophocles

  8. NN: a mathematical approach • From a computational point of view, ANNs represent a way to operate a non-linear functional mapping between an input and an output space. • This functional relation is expressed in an implicit way (via a combination of suitably weighted non-linear functions, in the case of MLF-NN) • ANNs are usually represented as groups of elementary computational units (neurons)performing simultaneously the same operations. • Types of NN differ in how neurons are grouped and how they operate

  9. output hidden input Multilayer feed-forward NN • Individual processing units are organized in three types of layer: input, hidden and output • All neurons within the same layer operate simultaneously y1 y2 y3 y4 x1 x2 x3 x4 x5

  10. x1 w1k w2k x2 zk  f() w3k hidden x3 input The artificial neuron x1 x2 x3 x4 x5

  11. z1 w1j output w2j z2 yj  f() w3j hidden z3 input The artificial neuron y1 y2 y3 y4 x1 x2 x3 x4 x5

  12. Training • Iterative variation of connection weights, to minimize an error criterion. • Usually, backpropagation algorithm is used:

  13. MLF class-modeling: what to do? • Model for each category has to be built using only training samples from that category • Suitable definition of category space

  14. Input Hidden Input x1 x2 x3 Xj xm Output value of hidden node 2 Output value of hidden node 1 Somewhere to start from When targets are equal to input values, hidden nodes could be thought of as a sort of non-linear principal components

  15. … and a first ending point • For each category a neural network model is computed providing the input vector also as desired target vector Ninp-Nhid-Ninp • Number of hidden layer is estimated by loo-cv (minimum reconstruction error in prediction) • The optimized model is then used to predict unknown samples: • Sample is presented to the network • Vector of predicted responses (which is an estimate of the original input vector) is computed • Prediction error is calculated and compared to the average prediction error for samples belonging to the category (as in SIMCA).

  16. if is lower than a predifined threshold, the sample is refused by the category model. NN-CM in practice • Separate category autoscaling

  17. A couple of examples

  18. The classical X-OR • 200 training samples: • 100 class 1 • 100 class 2 • 200 test samples: • 100 class 1 • 100 class 2 3 hidden neurons for each category

  19. Results • Sensitivity: • 100% class 1, 100% class2 • Specificity: • 75% class1 vs class 2 • 67% class2 vs class 1 • Prediction ability: • 87% class1 • 83% class2 • 85% overall • These results are significantly better than with SIMCA and UNEQ (specificities lower than 30% and classification slightly higher than 60%)

  20. A very small data-set: honey

  21. CM of honey samples • 76 samples of honey from 6 different botanical origins (honeydew, wildflower, sulla, heather, eucalyptus and chestnut) • 11-13 samples per class • 2 input variables: specific rotation; total acidity • Despite the small number of samples, a good NN model was obtained (2 hidden neurons for each class) • Possibility of drawing a Coomans’ plot

  22. Further work and Conclusions • A novel approach to class-modeling based on multilayer feed-forward NN was presented • Preliminary results seem to indicate its usefulness in cases where traditional class modeling fails • Effect of training set dimension should be further invetigated (our “small” data set was too good to be used for obtaining a definitive answer) • We are analyzing other “exotic” data sets for classification where traditional methods fail.

  23. Acknowledgements • Prof. Jure Zupan, Slovenia

More Related