### Kernel trick, cont’d

In 2-d:

(x)·(z)

= (1,2x1,2x2,x12,x22,2x1x2)·(1,2z1,2z2,z12,z22,2z1z2)

= 1 + 2x1z1 + 2x2z2 + x12z12 + x22z22 + 2x1x2z1z2

= (1 + x1z1 + x2z2)2

= (1 + x·z)2

In d dimensions:

(x)·(z)

= (1, 2x1,…,2xd, x12,…,xd2, 2x1x2,2x1x3,…,2xd-1xd)·

(1, 2z1,…,2zd, z12,…,zd2, 2z1z2,2z1z3,…, 2zd-1zd)

= (1 + x1z1 + x2z2 + … + xdzd)2

= (1 + x·z)2

Computing dot products in the 307,720-dimensional feature space takes time proportional to just 784, the original dimension!

Never need to write out (x).

Need w– but since it’s a linear combination of data points, just store the coefficients.