Download
1 / 14
140 likes | 250 Views
This introduction presents Locally Linear Embedding (LLE) in AI, a method for obtaining more useful representations of information. LLE is beneficial for subsequent operations like classification and pattern recognition, offering unsupervised learning and automatic recovery of hidden structures in data. It tackles the limitations of PCA and MDS by preserving local configurations of nearest neighbors in high-dimensional datasets. The LLE algorithm involves neighborhood search, constrained least square fits, and solving an eigenvalue problem to create a low-dimensional embedding. While LLE has advantages such as discovering nonlinear manifolds and global optimality, it also has limitations like sensitivity to outliers and requirements for smooth, densely sampled manifolds. Comparisons show how LLE differs from PCA and ISOMAP in finding the best-fit relationships in data representations.
E N D
Lightseminar: Learned Representation in AI An Introduction to Locally Linear Embedding Lawrence K. Saul Sam T. Roweis presented by Chan-Su Lee
Introduction • Preprocessing • Obtain more useful representation of information • Useful for subsequent operation for classification, pattern recognition • Unsupervised learning • Automatic methods to recover hidden structure • Density estimation • Dimensionality reduction
PCA & MDS • Principle Component Analysis:PCA • Linear projections of greatest variance from the top eigenvectors of the data covariance matrix • Multidimensional Scaling:MDS • Low dimensional embedding that best preserves pairwise distances between data points • Modeling of linear variabilities in high dimensional data • No local minima • Find linear subspace and cannot deal properly with data lying on nonlinear manifolds
Problem in PCA and MDS • Create distortion in local and global geometry • Map faraway data point to nearby point
LLE(Locally Linear Embedding) • Property • Preserving the local configurations of nearest neighbors • LLE • Local: only neighbors contribute to each reconstruction • Linear: reconstructions are confined to linear subspace • Assumption • Well-sampled data->locally linear patch of the manifold • d-dimensional manifold->2d neighbors
LLE Algorithm • Step1 Neighborhood search • Compute the neighbors of each data point • K nearest neighbors per data point
LLE Algorithm • Step2 Constrained Least Square Fits • Reconstruction Error if not neighbor -> Invariant to rotations, rescaling, and translation of that point and its neighbor
LLE Algorithm • Step3 Eigenvalue Problem • Reconstruction Error • -> centered at the origin • -> Avoid degenerate solution
Corners faces to the corners of its two dimensional embedding Embedding by LLE
LLE advantages • Ability to discover nonlinear manifold of arbitrary dimension • Non-iterative • Global optimality • Few parameters: K, d • O(DN2) and space efficient due to sparse matrix
LLE disadvantages • Requires smooth, non-closed, densely sampled manifold • Quality of manifold characterization dependent on neighborhood choice • Cannot estimate low dimensionality • Sensitive to outliers
Comparisons: PCA vs LLE vs Isomap • PCA: find linear subspace projection vectors to the data • LLE: find embedding coordinate vectors that best fit local neighborhood relationships • ISOMAP: find embedding coordinate vectors that preserve geodesic shortest distances
