Numerical computation of Non-Comm.
Download
1 / 16

Numerical computation of Non-Comm. VoI Metrics & Spectra of Random Graphs - PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on

Numerical computation of Non-Comm. VoI Metrics & Spectra of Random Graphs . Co-PI Raj Rao Nadakuditi University of Michigan. Research program Info-driven learning. Mission Information and Objectives. Non-commutative Info Theory. Info theoretic surrogates. Consensus learning.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Numerical computation of Non-Comm. VoI Metrics & Spectra of Random Graphs ' - wilma


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Numerical computation of Non-Comm. VoI Metrics & Spectra of Random Graphs

Co-PI Raj RaoNadakuditi

University of Michigan


Research program

Info-driven learning

Mission Information

and

Objectives

Non-commutative

Info Theory

Info theoretic

surrogates

Consensus

learning

Info-geometric

learning

Information-driven Learning

. Jordan (Lead); Ertin, Fisher,

Hero, Nadakuditi

Scalable, Actionable

VoI measures

Bounds, models and

learning algorithms


Eigen-analysis methods & apps.

  • Principal component analysis

    • Direction-finding (e.g. sniper localization)

    • Pre-processing/Denoising to SVM-based classification

    • (e.g. pattern, gait & face recognition)

    • Regression, Matched subspace detectors

    • Community/Anomaly detection in networks/graphs

  • Canonical Correlation Analysis

    • PCA-extension for fusing multiple correlated sources

  • LDA, MDS, LSI, Kernel(.) ++, MissingData(.)++

  • Eigen-analysis  Spectral Dim. Red. Subspace methods

  • Technical challenge:

    • Quantify eigen-VoI (Thrust 1) and Exploit quantified uncertainty (Thrust 2) for eigen-analysis based sensor fusion and learning


Role of Non-Comm. Info theory

  • For noisy, estimated subspaces, quantify:

    • Fundamental limits and phase transitions

    • Estimates of accuracy possibly, data-driven

    • Rates of convergence, learning rates

    • P-values

    • Impact of adversarial noise models

  • “Classical” info. measures in low-dim.-large sample regime

    • e.g. f-divergence, Shannon mutual info., Sanov’sthm.

      • vs.

  • Non-comm. info. measures in high-dim.-relatively-small-sample regime

    • Non-commutative analogs of above


  • Analytical signal-plus-noise model

    • Low dimensional (= k) latent signal model

    • Xnis n x m noise-only Gaussian matrix

    • c = n/m = # Sensors / # Samples

    • Theta ~ SNR


    Empirical subspaces are unequal

    • c = n/m = # Sensors / # Samples

    • Theta ~ SNR, X is Gaussian

    • Insight: Subspace estimates are biased!

      • “Large-n-large-m” versus “Small-n-large-m”


    A non-commutative VoImetric (beyond Gaussians)

    • Xnis n x m unitarily-invariant noise-only random matrix

    • Theorem [N. and Benaych-Georges, 2011]:

    • μ = Spectral measure of noise singular values

    • D = D-transform of μ “log-Fourier” transform in NCI


    Numerically computing D-transform

    • Desired:

      • Allow continuous and discrete valued inputs

      • O(n log n) where n is number of singular values

      • Numerically stable


    Empirical VoI quantification

    • Based on an eigen-gap based segment, compute non-commVoI subspaces


    Accomplishment - I

    • Uk are Chebyshev polynomials

    • Series coefficients computed via DCT in O(n log n)

    • Closed-form G transform (and hence D transform) series expansion!

    • “Numerical computation of convolutions in free probability theory” (with Sheehan Olver)


    Broader Impact

    • For noisy, estimated subspaces, quantify:

      • Fundamental limits and phase transitions

      • Estimates of accuracy possibly, data-driven

      • Rates of convergence, learning rates

      • P-values

      • Impact of adversarial noise models

      • Impact of finite training data

    • Facilitate fast, accurate performance prediction for eigen-methods!

    • Transition: MATLAB toolbox


    Spectra of Networks

    • Role of spectra of social and related networks:

      • Community structure discovery

      • Dynamics

      • Stability

    • Open problem: Predict graph spectra given degree sequence

    • Broader Impact: ARL CTA & ITA, ARO MURI


    Non. Comm. Prob. for Network Science

    • Role of spectra of social and related networks:

      • Community structure discovery

      • Dynamics

      • Stability

    • OpenSolved problem: Predict spectra of a graph given expected degree sequence

    • Answer: Free multiplicative convolution of degree sequence with semi-circle

    • “Spectra of graphs with expected degree sequence” (with Mark Newman)


    Accomplishment - II

    • Predicting spectra (numerical free convolution – Accomplishment I)

    • “When is a hub not a hub (spectrally)?”

    • New phenomena, new VoI analytics


    Phase transition in comm. detection

    • Unidentifable: If cin – cout < 2

    • cin = Avg. degree “within”; cout= Avg. degree “without”


    Relation to other research thrusts

    • Accomplishments

      • Numerical computation of Non-Commconvolutions

      • Predicting spectra of complicated networks

    • Impact

      • Information fusion

        • Numerical computation of Non-Comm. Metrics

        • Performance prediction

        • New VoI analytics for networks

        • Predicting graph spectra from degree sequence

      • Information exploitation

        • Selective fusion of subspace information


    ad