1 / 27

Modeling Ultra-high Dimensional Feature Selection as a Slow Intelligence System

This project outlines a framework for modeling ultra-high dimensional feature selection as a slow intelligence system. It explores iterative feature selection methods and introduces a time controller based on a Petri Net to control the process. The project aims to design a system that can effectively select important features in high-dimensional datasets.

chismd
Download Presentation

Modeling Ultra-high Dimensional Feature Selection as a Slow Intelligence System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Modeling Ultra-high Dimensional Feature Selection as a Slow Intelligence System Wang Yingze CS 2650 Project

  2. Outline Introduction Introduction Iterative feature selection Framework of Slow Intelligence System Tasks for project Midway results

  3. Introduction • UltraHigh-dimensional variable selection is the hot topic in statistics and machine learning. • Model relationship between one response and associated features , based on a sample of size n.

  4. Application • Associated studies between phenotypes and SNPs. • Gene selection or disease classification in bioinformatics. each patient’s data with p genes n Patients’ degree of disease sickness Important genes selected one Gene expression level

  5. Challenges • Dimensionality grows rapidly with interactions of the features Portfolio selection and networking modeling: 2000 stocks involves over 2 millions unknown parameters in the covariance matrix. Protein-protein interaction: the sample size may be in the order of thousands, but the number of features can be in the order of millions. • To construct effective method to learn relationship between features and response in high dimension for scientific purposes.

  6. Outline Introduction Iterative feature selection Iterative feature selection Framework of Slow Intelligence System Tasks for project Midway results

  7. Existing methods • LASSO :L1 regularization linear regression • Forward regression:sequentially add variables • Backward regression: start with them all then delete them on the bases of smallest change in • Stepwise regression: at each step one can be entered (on basis of greatest improvement in but one also may be removed if the change (reduction) in is not significant. • Least-angle regression: estimated parameters are increased in a direction equiangular to each one's correlations with the residual.

  8. State-of-the-Art Approach • Interactive feature selection method proposed by Jianqing Fan in Princeton University • “Ultrahigh dimensional feature selection: beyond the linear model” • Contribution: • Ultrahigh dimensional data • Accuracy • slow

  9. Step 1: Large-scale screening • Apply Pearson correlation and ranking to pick a set

  10. Step 2 : Moderate-scale selection • Employ an existing regression method to select a subset • of these indices .

  11. Step 3: Large-scale screening • Adding other features one each time with to the regression model:

  12. Step 3 (con’t) • Ranking j features according to , select the top numbers of features. And add to forming the new feature set • Repeats Steps 2-3, select new from , then form new • until

  13. Outline Introduction Iterative feature selection Framework of Slow Intelligence System Framework of Slow Intelligence System Tasks for project Midway results

  14. Slow Intelligence System • “A General Framework for Slow Intelligence Systems”, by S.K. Chang, International Journal of Software Engineering and Knowledge Engineering

  15. Time Controller • Slow decision cycle(s) to complement quick decision cycle(s): SIS possesses at least two decision cycles. Therefore, Slow Intelligence Systems work usually correctly but not always fast. • Time Controller Design • PanicButton • Petri-net model

  16. Motivation • “Modeling Human Intelligence as A Slow Intelligence System” • by Tiansi Dong, DMS2010 • SIS forobject mapping between scenes • Two object tracing results due to two different priorities • 1. Priority on spatial changes (minimal spatial changes) • 2. Priority on object categories (objects are mapped within same categories)

  17. SIS1 for Object tracing (priority on spatial changes) • Enumerate all possible mapping • Elimination and concentration the mapping with the minimal spatial changes

  18. SIS2 for Object tracing (priority onobject category) • Enumerate all possible mapping • Elimination and concentration the mapping with the same category

  19. Outline Introduction Iterative feature selection Framework of Slow Intelligence System Tasks for project Tasks for project Midway results

  20. Task one • Modeling Ultra-high Dimensional Feature Selection as a Slow Intelligence System • Use SIS to model Iterative feature selection method to five phases: Enumeration, Elimination, Adaptation, Propagation, Concentration. • The whole SIS system contains additional Sub-SIS system. • Represent it in Mathematical formulation

  21. Task two • Design time controller in term of Petri Net and introduce Knowledge base • Time controller controls the time to evoke each phase of SIS. • Knowledge base contains five different moderate-scale selection algorithms. KB can be changed and updated in the slow cycle. • Represent Time controller in Petri Net using ReNew Editor

  22. Future work • Experiment: Use some real data (colon cancer data) to do the experiment and compare the results with some existing feature selection method like LASSO, forward, backward regression, etc. Weka: Demo • I will use some visualization tool to visualize the result and the process of feature selection.

  23. Outline Introduction Iterative feature selection Framework of Slow Intelligence System Tasks for project Midway results Midway results

  24. Diagram: Main SIS System

  25. Diagram: Sub SIS system

  26. Diagram: Petri-Net Model

  27. Thank You! Q & A

More Related