1 / 45

Multimedia Databases

Multimedia Databases. SVD II. SVD - Detailed outline. Motivation Definition - properties Interpretation Complexity Case studies SVD properties More case studies Conclusions. SVD - detailed outline. ... Case studies SVD properties more case studies google/Kleinberg algorithms

mateja
Download Presentation

Multimedia Databases

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimedia Databases SVD II

  2. SVD - Detailed outline • Motivation • Definition - properties • Interpretation • Complexity • Case studies • SVD properties • More case studies • Conclusions

  3. SVD - detailed outline • ... • Case studies • SVD properties • more case studies • google/Kleinberg algorithms • Conclusions

  4. Optimality of SVD Def: TheFrobenius norm of a n x m matrix M is (reminder) The rank of a matrix M is the number of independent rows (or columns) of M Let A=ULVT and Ak = UkLk VkT (SVD approximation of A) Theorem: [Eckart and Young] Among all n x m matrices C of rank at most k, we have that:

  5. SVD - Other properties - summary • can produce orthogonal basis • can solve over- and under-determined linear problems (see C(1) property) • can compute ‘fixed points’ (= ‘steady state prob. in Markov chains’) (see C(4) property)

  6. SVD -outline of properties • (A): obvious • (B): less obvious • (C): least obvious (and most powerful!)

  7. Properties - by defn.: A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] A(1): UT [r x n] U [n x r ] = I [r x r ] (identity matrix) A(2): VT [r x n] V [n x r ] = I [r x r ] A(3): Lk = diag( l1k, l2k, ... lrk ) (k: ANY real number) A(4): AT= V L UT

  8. Less obvious properties A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] B(1): A[n x m] (AT) [m x n] = ??

  9. Less obvious properties A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] B(1): A[n x m] (AT) [m x n] = UL2UT symmetric; Intuition?

  10. Less obvious properties A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] B(1): A[n x m] (AT) [m x n] = UL2UT symmetric; Intuition? ‘document-to-document’ similarity matrix B(2): symmetrically, for ‘V’ (AT) [m x n] A[n x m] = VL2VT Intuition?

  11. Less obvious properties B(3): ((AT) [m x n] A[n x m] )k= VL2kVT and B(4): (ATA )k ~ v1l12kv1T for k>>1 where v1: [m x 1] first column (eigenvector) of V l1: strongest eigenvalue

  12. Less obvious properties B(4): (ATA )k ~ v1l12kv1T for k>>1 B(5): (ATA )k v’ ~ (constant) v1 ie., for (almost) any v’, it converges to a vector parallel to v1 Thus, useful to compute first eigenvector/value (as well as the next ones…)

  13. Less obvious properties - repeated: A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] B(1): A[n x m] (AT) [m x n] = UL2UT B(2): (AT) [m x n] A[n x m] = VL2VT B(3): ((AT) [m x n] A[n x m] )k= VL2kVT B(4): (ATA )k ~ v1l12k v1T B(5): (ATA )k v’ ~ (constant) v1

  14. Least obvious properties A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] C(1): A[n x m]x [m x 1] = b [n x 1] let x0 = VL(-1)UTb if under-specified, x0 gives ‘shortest’ solution if over-specified, it gives the ‘solution’ with the smallest least squares error

  15. Least obvious properties Illustration: under-specified, eg [1 2] [w z] T = 4 (ie, 1 w + 2 z = 4) z shortest-length solution x0 all possible solutions 2 1 1 2 3 4 w

  16. Least obvious properties - cont’d A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] C(2): A[n x m]v1 [m x 1] = l1 u1[n x 1] where v1 , u1 the first (column) vectors of V, U. (v1== right-eigenvector) C(3): symmetrically: u1T A = l1 v1T u1 == left-eigenvector Therefore:

  17. Least obvious properties - cont’d A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] C(4): AT Av1= l12 v1 (fixed point - the usual dfn of eigenvector for a symmetric matrix)

  18. Least obvious properties - altogether A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] C(1): A[n x m]x [m x 1] = b [n x 1] then, x0 = VL(-1)UTb: shortest, actual or least-squares solution C(2): A[n x m]v1 [m x 1] = l1 u1[n x 1] C(3): u1T A = l1 v1T C(4): AT Av1= l12 v1

  19. Properties - conclusions A(0): A[n x m] = U[ n x r ]L[ r x r ]VT[ r x m] B(5): (ATA )k v’ ~ (constant) v1 C(1): A[n x m]x [m x 1] = b [n x 1] then, x0 = VL(-1)UTb: shortest, actual or least-squares solution C(4): AT Av1= l12 v1

  20. SVD - detailed outline • ... • Case studies • SVD properties • more case studies • Kleinberg/google algorithms • Conclusions

  21. Kleinberg’s Algorithm • Main idea: In many cases, when you search the web using some terms, the most relevant pages may not contain this term (or contain the term only a few times) • Harvard : www.harvard.edu • Search Engines: yahoo, google, altavista • Authorities and hubs

  22. Kleinberg’s algorithm • Problem dfn: given the web and a query • find the most ‘authoritative’ web pages for this query Step 0: find all pages containing the query terms (root set) Step 1: expand by one move forward and backward (base set)

  23. Kleinberg’s algorithm • Step 1: expand by one move forward and backward

  24. Kleinberg’s algorithm • on the resulting graph, give high score (= ‘authorities’) to nodes that many important nodes point to • give high importance score (‘hubs’) to nodes that point to good ‘authorities’) hubs authorities

  25. Kleinberg’s algorithm observations • recursive definition! • each node (say, ‘i’-th node) has both an authoritativeness score ai and a hubness score hi

  26. Kleinberg’s algorithm Let E be the set of edges and A be the adjacency matrix: the (i,j) is 1 if the edge from i to j exists Let h and a be [n x 1] vectors with the ‘hubness’ and ‘authoritativiness’ scores. Then:

  27. Kleinberg’s algorithm Then: ai = hk + hl + hm that is ai = Sum (hj) over all j that (j,i) edge exists or a = ATh k i l m

  28. Kleinberg’s algorithm symmetrically, for the ‘hubness’: hi = an + ap + aq that is hi = Sum (qj) over all j that (i,j) edge exists or h = Aa i n p q

  29. Kleinberg’s algorithm In conclusion, we want vectors h and a such that: h = Aa a = ATh Recall properties: C(2): A[n x m]v1 [m x 1] = l1 u1[n x 1] C(3): u1T A = l1 v1T

  30. Kleinberg’s algorithm In short, the solutions to h = Aa a = ATh are the left- and right- eigenvectors of the adjacency matrix A. Starting from random a’ and iterating, we’ll eventually converge (Q: to which of all the eigenvectors? why?)

  31. Kleinberg’s algorithm (Q: to which of all the eigenvectors? why?) A: to the ones of the strongest eigenvalue, because of property B(5): B(5): (ATA )k v’ ~ (constant) v1

  32. Kleinberg’s algorithm - results Eg., for the query ‘java’: 0.328 www.gamelan.com 0.251 java.sun.com 0.190 www.digitalfocus.com (“the java developer”)

  33. Kleinberg’s algorithm - discussion • ‘authority’ score can be used to find ‘similar pages’ to page p (how?) • closely related to ‘citation analysis’, social networs / ‘small world’ phenomena

  34. google/page-rank algorithm • closely related: The Web is a directed graph of connected nodes • imagine a particle randomly moving along the edges (*) • compute its steady-state probabilities. That gives the PageRank of each pages (the importance of this page (*) with occasional random jumps

  35. PageRank Definition • Assume a page A and pages P1, P2, …, Pm that point to A. Let d is a damping factor. PR(A) the pagerank of A. C(A) the out-degree of A. Then:

  36. google/page-rank algorithm • Compute the PR of each page~identical problem: given a Markov Chain, compute the steady state probabilities p1 ... p5 2 1 3 4 5

  37. Computing PageRank • Iterative procedure • Also, … navigate the web by randomly follow links or with prob p jump to a random page. Let A the adjacency matrix (n x n), di out-degree of page i Prob(Ai->Aj) = pn-1+(1-p)di–1Aij A’[i,j] = Prob(Ai->Aj)

  38. google/page-rank algorithm • Let A’ be the transition matrix (= adjacency matrix, row-normalized : sum of each row = 1) 2 1 3 = 4 5

  39. google/page-rank algorithm • A p = p A p = p 2 1 3 = 4 5

  40. google/page-rank algorithm • A p = p • thus, p is the eigenvector that corresponds to the highest eigenvalue (=1, since the matrix is row-normalized)

  41. Kleinberg/google - conclusions SVD helps in graph analysis: hub/authority scores: strongest left- and right- eigenvectors of the adjacency matrix random walk on a graph: steady state probabilities are given by the strongest eigenvector of the transition matrix

  42. Conclusions • SVD: a valuable tool • given a document-term matrix, it finds ‘concepts’ (LSI) • ... and can reduce dimensionality (KL) • ... and can find rules (PCA; RatioRules)

  43. Conclusions cont’d • ... and can find fixed-points or steady-state probabilities (google/ Kleinberg/ Markov Chains) • ... and can solve optimally over- and under-constraint linear systems (least squares)

  44. Brin, S. and L. Page (1998). Anatomy of a Large-Scale Hypertextual Web Search Engine. 7th Intl World Wide Web Conf. Chen, C. M. and N. Roussopoulos (May 1994). Adaptive Selectivity Estimation Using Query Feedback. Proc. of the ACM-SIGMOD , Minneapolis, MN. References

  45. Kleinberg, J. (1998). Authoritative sources in a hyperlinked environment. Proc. 9th ACM-SIAM Symposium on Discrete Algorithms. Press, W. H., S. A. Teukolsky, et al. (1992). Numerical Recipes in C, Cambridge University Press. References cont’d

More Related