Extending metric multidimensional scaling with bregman divergences
Download
1 / 27

Extending metric multidimensional scaling with Bregman divergences - PowerPoint PPT Presentation


  • 99 Views
  • Uploaded on

Extending metric multidimensional scaling with Bregman divergences. Mr. Jigang Sun Supervisor: Prof. Colin Fyfe Nov 2009. Multidimensional Scaling(MDS).

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Extending metric multidimensional scaling with Bregman divergences' - mervin


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Extending metric multidimensional scaling with bregman divergences

Extending metric multidimensional scaling with Bregman divergences

Mr. Jigang Sun

Supervisor: Prof. Colin Fyfe

Nov 2009


Multidimensional scaling mds
Multidimensional Scaling(MDS)

  • A group of information visualisation methods that projects data from high dimensional space, to a low dimensional space, often two or three dimensions, keeping inter-point dissimilarities (e.g. distances) in low dimensional space as close as possible to the original dissimilarities in high dimensional space. When Euclidean distances are used, it is Metric MDS.


An example
An example

low dimensional space

/latent space/output space

high dimensional space

/data space/input space

basic MDS


Basic mds
Basic MDS

  • We minimise the stress function

data space

Latent space


Sammon mapping 1969
Sammon Mapping (1969)

Focuses on small distances: for the same error, the smaller distance is given bigger stress, thus on average the small distances are mapped more accurately than long distances. Small neighbourhoods are well preserved.


Bregman divergence
Bregman divergence

is the Bregman divergence between p and q based on strictly convex

function, F. Intuitively, the difference between the value of F at point p and the value of the first-order Taylor expansion of F around point q evaluated at point p.


Bregman divergence

  • When F is in one variable, the Bregman Divergence is truncated Taylor series

  • A useful property for MDS: Non-negativity:

  • If is a function in p, p approaches q when it is minimised.


Mds using bregman divergence
MDS using Bregman divergence

  • Bregmanised MDS

  • Equivalent Expression: residual Taylor series


Basic mds is a special bmmds
Basic MDS is a special BMMDS

  • Base convex function is chosen as

  • And higher order derivatives are

  • So

  • Is derived as


Example 2 extended sammon
Example 2: Extended Sammon

  • Base convex function

  • This is equivalent to

  • The Sammon mapping is rewritten as


Sammon and extended sammon
Sammon and Extended Sammon

  • The common term

  • The Sammon mapping is considered to be an approximation to the Extended Sammon mapping using the common term.

  • The Extended Sammon mapping will do more adjustments on the basis of the higher order terms.



At a glance
At a glance

  • Basic MDS captures the global curve, but poorly differentiates local points of same X and Y coordinate but different Z coordinate.

  • The Sammon mapping does better than BasicMDS.

  • The Extended Sammon mapping is the best.



Distance preservation

  • Horizontal axis: mean distances in data space, 40 sets.

  • Vertical axis: relative mean distances in latent space.

  • Sammon is better than BasicMDS, Extended Sammon is better than Sammon:

  • Small distances are mapped closer to their original value in data space; long distances are mapped longer.



Relative standard deviation1
Relative standard deviation

  • On short distances, Sammon has smaller variance than BasicMDS, Extended Sammon has smaller variance than Sammon, i.e. control of small distances is enhanced.

  • Large distances are given more and more freedom in the same order as above.


Lcmc local continuity meta criterion l chen 2006
LCMC: local continuity meta-criterion (L. Chen 2006)

  • A common measure assesses projection quality of different MDS methods.

  • In terms of neighbourhood preservation.

  • Value between 0 and 1, the higher the better.



Stress comparison between sammon and extended sammon
Stress comparison between Sammon and Extended Sammon


Stress comparison between sammon and extended sammon1
Stress comparison between Sammon and Extended Sammon

  • For the ExtendedSammon, a shorter distance error (e.g. if Dij-Lij=2) in latent space is penalized more than a longer distance error (e.g. if Dij – Lij =-2)in latent space.



Stress formation by terms
Stress formation by terms

  • Stress coming from the term of the Sammon mapping is the largest. It is the main part of stress.

  • However, for small distances, the contribution from other terms is not negligible.


Openbox sammon and firstgroup
OpenBox, Sammon and FirstGroup


Secondgroup on openbox
SecondGroup on OpenBox


Future work
Future work

  • Combining two opposite strategies for choosing base convex functions.

  • Right Bregman divergences is one kind of CCA.


Conclusion
Conclusion

  • Applied Bregman divergences to multidimensional scaling.

  • Shown that basic MMDS is a special case and Sammon mapping approximates a BMMDS.

  • Improved upon both with 2 families of divergences.

  • Shown results on two artificial data sets.


ad