Multilayer feed-forward artificial neural networks for Class-modeling

1. Multilayer feed-forwardartificial neural networks for Class-modeling F. Marini, A. Magrì, R. Bucci Dept. of Chemistry - University of Rome “La Sapienza”

2. The starting question…. Despite literature on NNs has increased significantly, no paper considers the possibility of performing class-modeling

3. class modeling: what…. • Class modeling considers one class at a time • Any object can then belong or not to that specific class model • As a consequence, any object can be assigned to only one class, to more than one class or to no class at all classification class modeling

4. less equivocal answer to the question: “are the analytical data compatible with the product being X as declared?” …..and why • Flexibility • Additional information: • sensitivity: fraction of samples from category X accepted by the model of category X • specificity: fraction of samples from category Y (or Z, W….) refused by the model of category X • No need to rebuild the existing models each time a new category is added.

5. A first step forward • A particular kind of NN, after suitable modifications could be used for performing class-modeling (Anal. Chim. Acta, 544 (2005), 306) • Kohonen SOM • Addition of dummy random vectors to the training set • Computation of a suitable (non-parametric) probability distribution after mapping on the 2D Kohonen layer. • Definition of the category space based on this distribution

6. In this communication… …The possibility of using a different type of neural network (multilayer feed-forward) to operate class-modeling is studied • How to? • Examples

7. Just a few words about NN Polla ta deina kouden an- qropou deinoteron pelei. Sophocles

8. NN: a mathematical approach • From a computational point of view, ANNs represent a way to operate a non-linear functional mapping between an input and an output space. • This functional relation is expressed in an implicit way (via a combination of suitably weighted non-linear functions, in the case of MLF-NN) • ANNs are usually represented as groups of elementary computational units (neurons)performing simultaneously the same operations. • Types of NN differ in how neurons are grouped and how they operate

9. output hidden input Multilayer feed-forward NN • Individual processing units are organized in three types of layer: input, hidden and output • All neurons within the same layer operate simultaneously y1 y2 y3 y4 x1 x2 x3 x4 x5

10. x1 w1k w2k x2 zk  f() w3k hidden x3 input The artificial neuron x1 x2 x3 x4 x5

11. z1 w1j output w2j z2 yj  f() w3j hidden z3 input The artificial neuron y1 y2 y3 y4 x1 x2 x3 x4 x5

12. Training • Iterative variation of connection weights, to minimize an error criterion. • Usually, backpropagation algorithm is used:

13. MLF class-modeling: what to do? • Model for each category has to be built using only training samples from that category • Suitable definition of category space

14. Input Hidden Input x1 x2 x3 Xj xm Output value of hidden node 2 Output value of hidden node 1 Somewhere to start from When targets are equal to input values, hidden nodes could be thought of as a sort of non-linear principal components

15. … and a first ending point • For each category a neural network model is computed providing the input vector also as desired target vector Ninp-Nhid-Ninp • Number of hidden layer is estimated by loo-cv (minimum reconstruction error in prediction) • The optimized model is then used to predict unknown samples: • Sample is presented to the network • Vector of predicted responses (which is an estimate of the original input vector) is computed • Prediction error is calculated and compared to the average prediction error for samples belonging to the category (as in SIMCA).

16. if is lower than a predifined threshold, the sample is refused by the category model. NN-CM in practice • Separate category autoscaling

17. A couple of examples

18. The classical X-OR • 200 training samples: • 100 class 1 • 100 class 2 • 200 test samples: • 100 class 1 • 100 class 2 3 hidden neurons for each category

19. Results • Sensitivity: • 100% class 1, 100% class2 • Specificity: • 75% class1 vs class 2 • 67% class2 vs class 1 • Prediction ability: • 87% class1 • 83% class2 • 85% overall • These results are significantly better than with SIMCA and UNEQ (specificities lower than 30% and classification slightly higher than 60%)

20. A very small data-set: honey

21. CM of honey samples • 76 samples of honey from 6 different botanical origins (honeydew, wildflower, sulla, heather, eucalyptus and chestnut) • 11-13 samples per class • 2 input variables: specific rotation; total acidity • Despite the small number of samples, a good NN model was obtained (2 hidden neurons for each class) • Possibility of drawing a Coomans’ plot

23. Further work and Conclusions • A novel approach to class-modeling based on multilayer feed-forward NN was presented • Preliminary results seem to indicate its usefulness in cases where traditional class modeling fails • Effect of training set dimension should be further invetigated (our “small” data set was too good to be used for obtaining a definitive answer) • We are analyzing other “exotic” data sets for classification where traditional methods fail.

24. Acknowledgements • Prof. Jure Zupan, Slovenia