The Perceptron Algorithm (dual form)

1. The Perceptron Algorithm (dual form)

2. The perceptron algorithm works by • Adding missclassified positive training examples or • Subtracting misclassified negative ones to an initial arbitrary weight vector • We assumed that the initial weight vector is the zero vector, and so te final hyothesis will be a • Linear combination of the training points: The number of times misclassification of Xi has Caused the weight to be updated. The part of Perceptron algorithm (Primal form)

3. The number of times misclassification of xi has Caused the weight to be updated. • Points that have caused fewer mistakes will have smaller αi, • whereas difficult points will have large values • This quantity is sometimes referred to as the embedding strength of the pattern xi

4. Once a sample S has been fixed, one can think of the vector αas alternative representation of the hypothesis in different of dual coordinates. • This expansion is not unique: different αcan correspond to the same hypothesis w • One can also regard αias an indication of the information content of the example xi

5. The decision function can be rewritten in dual coordinates as follows: 結論: 原本是 x 直接與 w 作內積, 現在變成直接與 training example 作內積

6. Update w Update αi (每個 x 都有自己的 α) The Perceptron algorithm (primal form) The Perceptron algorithm (dual form)

7. The dual form’s property • The points that have larger αi can be used to rank the data according to their information content • Remark 2.10 Since the number of updates equals the number of mistakes and each update causes 1 to be added to exactly one of its components, the 1-norm of the vector αsatisifies 結論: the 1-norm of α as the complexity of the target concept in the dual representation

8. Remark 2.11 • The training data only enter the algorithm through the entries of the matrix G, known as the Gram matrix

9. Dual Representation of Linear Machines • The data only appear through entries in the Gram matrix and never through their individual attributes • In the decision function, it is only the inner products of the data with the new test point that are needed