1 / 23

Kai Lei 1 , Meng Qin 1 , Bo Bai 2,* , Gong Zhang 2

NetAI 2018 , Budapest, Hungary. Adaptive Multiple Non-negative Matrix Factorization for Temporal Link Prediction in Dynamic Networks. Kai Lei 1 , Meng Qin 1 , Bo Bai 2,* , Gong Zhang 2. 1 ICNLAB, SECE, Peking University Shenzhen Graduate School

millerbenjamin
Download Presentation

Kai Lei 1 , Meng Qin 1 , Bo Bai 2,* , Gong Zhang 2

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. NetAI 2018, Budapest, Hungary Adaptive Multiple Non-negative Matrix Factorization for Temporal Link Prediction in Dynamic Networks Kai Lei1, Meng Qin1, Bo Bai2,*, Gong Zhang2 1ICNLAB, SECE, Peking University Shenzhen Graduate School 2Future Network Theory Lab, 2012 Labs, Huawei Technologies Co. Ltd 2018-8-24 (Friday)

  2. Outline Dynamics Prediction Task of Various Network Systems Generally Formulate！ Temporal Link Prediction Problem • Motivation • Problem Definition • Methodology • Experimental Evaluation • Conclusion

  3. Motivation • Dynamics of network systems • a significant factor that hinders systems' performance • The prediction of mobility, topology & traffic • an effective technique to tackle such dynamics problem • e.g. mobility prediction in cellular networks [N. B. Prajapati, et al. '18] • e.g. traffic prediction in data center networks [A. Mozo, et al. '18] • etc.

  4. Motivation(Cont) Only focus on a specific application scenario! But cannot be generalized to other different scenarios! • Dynamics prediction of network systems • e.g. [A. Mozo, et al. '18] CNN based model • forecast short-term traffic load in data center networks • e.g. [L. Nie, et al. '18] deep belief neural network & compressive sensing • predict traffic in wireless mesh backbone networks • e.g. [N. B. Prajapati. '18] hidden Markov model • predict users’ location in mobile cellular networks

  5. Motivation(Cont) Most prediction tasks of mobility, traffic & topology can be generally formulated as the temporal link prediction problem! if d(vi, vj)≤δ then Aij=Aji=1 Data trans. relation si sj si sj vi vj vi vj Edge Distance Edge (Traffic) (Edge Wights) d(vi, vj) Switch Node Vehicle Node Data center networks Vehiclemobility networks • Constructing Dynamic Networks • Abstract the entities and the corresponding relations • e.g. Data center networks; Vehicle mobility networks

  6. Aτ-l Aτ-l+1 Aτ-1 Aτ … Problem Definition Ãτ+1 Current net. snapshot Next net. snapshot (Historical net. snapshot) Prediction result The model Adjacency matrix • The temporal link prediction problem • (Consider undirected network with a fixed node set.) • Definition:

  7. Low-dim. Hidden Space Methodology At N×N Yt N×K XtT K×N (K<N) ≈ × • NMF-based network embedding • Non-negative Matrix Factorization (NMF) [D. Lee, Nature ‘99] • At : Adjacency matrix; • Xt≥0: Network represent. matrix; • Yt≥0: Auxiliary matrix. • Objective Function: • Introduces a low-dimensional hidden space (K-dimension) • Preserve thetopological characteristic of single snapshotAt

  8. Methodology(Cont) RespectiveYt for single NMF comp. Encodes the hid. info. of each single network snapshot SharedX for all NMF Comp. Encodes the hid. info. of the dynamic network (with mult. successive snapshot) • Unified temporal link prediction model • Define NMF componentt: • Corresponding to a single network snapshotAt • Use linear combination of NMF components {τ-l, …,τ-1,τ}

  9. Methodology(Cont) More contribution Less contribution Aτ-l Aτ-l+1 Aτ-1 Aτ Ãτ+1 … Exponent penalty for time factor Param. to control NMF comp. t • Unified temporal link prediction model (Cont) • Use ρt∈[0, 1] to adjust NMF componentst’s contribution • Assumption about the time factor • Utilize weighted exponential decaying penalty

  10. Methodology(Cont) Sim. between single snap. & the dynamic net. Euclidean dist.-induced similarity Adaptive param. Hidden info. of net. snapshott Hidden info.of the dynamic network (Max-min normalization) • Unified temporal link prediction model (Cont) • Reduce the complexity / Number of param. • Introduce the adaptive parameterρt =ρt(X, Xt) • Adaptively control the NMF comp. t

  11. Methodology(Cont) For ada. param. & init. of X For init.of {Yt(0)} NMF Comp. t (τ-l ≤ t ≤ τ) • The Adaptive Multiple NMF. (AM-NMF) model • Objective function(in the s-th iteration): • Properly initialize {X,Yτ-l,…,Yτ} • Utilize certain rules to continuously update{X,Yτ-l,…,Yτ} until converge • Initialization • X(0)←Xτ* (Solution of NMF comp.τ) • Yt(0) ←Yt* (Solution of NMF comp.t)

  12. Methodology(Cont) • The AM-NMF model (Cont) • Construct the prediction resultÃτ+1 • Conduct the inverse process of NMF from the hidden space • The base method • Directly use the solution {X, Yτ} • Without additional parameters • Katz-refining (KR) • Use Katz-index[Leo Katz. ‘53] to refine the result • Need to adjust param. {β, θ};

  13. Methodology(Cont) • Superiority of the AM-NMF model • Reduce the number of param. to be adjusted • ρtadaptively adjust the contribution of NMF comp. t • With only one param. α to be adjusted • Consider the intrinsic correlation • Between single net. snap. & dynamic net. • Can be easily extended to a hybrid network embedding model • That integrates other heterogeneous info. • e.g. higher-order topology (i.e. motif) [J. Leskovec et al. Science '16] • e.g. node attributes • Only need to determine the corresponding NMF comp.

  14. Methodology(Cont) standard NMF process[D. Lee, Nature ‘99] Input: {Aτ-l, …,Aτ}, {Xτ-l*,Yτ-l*,…,Xτ-1*,Yτ-1*} Get the solution of NMF comp. τ {Xτ*,Yτ*} • To partly avoid local minima solution • Take such init. step at least 10 times Initialize {X,Yτ-l, …,Yτ} Alternatively conduct Y-Process and X-Process until converge Output: Ãτ+1 • The AM-NMF Algorithm • For current time slice τ • Assume solutions of previous NMF comp. {τ-l, …,τ-1} have been saved

  15. Methodology(Cont) Obj. function Partial derivativewith respect to Yt Addictive rule (Gradient Descend) Multiplicative rule • The AM-NMF Algorithm(Cont) • Y-Process: UpdateYt (τ-l ≤ t ≤ τ) with {Yp, X} (p ≠ t) fixed

  16. Methodology(Cont) Obj. function Partial derivativewith respect to Yt Multiplicative updating rule • The AM-NMF Algorithm (Cont) • X-Process: UpdateX with {Yt} (τ-l ≤ t ≤ τ) fixed

  17. Experimental Evaluation N: num. of nodes; T: num of time slices K: dim. of hid. space we set • Datasets • KAIST: Human mobility network (position) • BJ-Taxi: Vehicle mobility network (position) • UCSB: Wireless mesh network (link quality) • NumFab: Center data network (flow) • Preprocess – Extract multiple successive network snapshots • Unweighted network: KAIST, BJ-Taxi • Calculate distance between each pair of nodes; Aij=Aji=1, if dij=dji ≤δ; • Weighted network: UCSB, NumFab • Use link quality/flow as the corresponding weighted edge.

  18. Conv. Curve of the (AM-NMF) Obj. Function Conv. Curve of the Adaptive Param. Experimental Evaluation(Cont) • Parameter Analysis • Convergence of the AM-NMF algorithm with varied param. ρt • Prediction process of one net. snpashot of KAIST • (Uiformly set l = 10) • Variation cure of (AM-NMF) obj. function & all the adaptive param.

  19. Experimental Evaluation(Cont) True Positive Rate False Positive Rate • Performance Evaluation • Evaluation Metric • Prediction of unweighted networks • Area Under the ROC Curve (AUC) • (ROC: Receiver Operating Characteristic) • Prediction of weighted networks • Avg. Error Rate / Mean Square Error (MSE)

  20. Experimental Evaluation(Cont) • Comparative methods • Collapsed network based: WCT, ED, SVD • NMF based: SNMF-FC, GrNMF • Performance Evaluation (Cont) • Prediction of unweighted networks • Datasets: KAIST, BJ-Taxi • Metric: (Avg.) AUC • AM-NMF with Katz-refining (KR): best • AM-NMF without KR: second-best • Prediction of weighted networks • Datasets: UCSB, NumFab • Metric: (Avg.) MSE • AM-NMF without KR: best

  21. Conclusion • Generally formulate the dynamics prediction of network systems • As the temporal link prediction problem • Propose a novel AM-NMF model • Introduce adaptive param. to reduce number of param. to be adjusted • Consider the intrinsic correlation between single snapshot and dynamic net. • Can be extended to a hybrid network embed. model integrates other info. • e.g. higher-order topo. (motif), node attribute • Derive the solving strategy with ensured convergence

  22. Conclusion(Cont) • Future work • The non-linear characteristics of dynamic networks • The effect of window sizel • The sampling frequency of network snapshots • The challenging case with varied node set ! • For weighted dynamic networks • Wide-value range problem of edge weights • Sparsity of edge weights

  23. NetAI 2018, Budapest, Hungary Adaptive Multiple Non-negative Matrix Factorization for Temporal Link Prediction in Dynamic Networks Thank You Very Much! Q&A Meng Qin (megnqin_az@foxmail.com)

More Related