data exploration with davis n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Data Exploration with DAVIS PowerPoint Presentation
Download Presentation
Data Exploration with DAVIS

Loading in 2 Seconds...

play fullscreen
1 / 31

Data Exploration with DAVIS - PowerPoint PPT Presentation


  • 105 Views
  • Uploaded on

Moon HUH 1 , KwangRyeol SONG 2 , YoungSuk PARK 1, KyungWook Shim 1 Sungkyunkwan University, Seoul, Korea 2 Kwansei Research Institute , Seoul, Korea. Data Exploration with DAVIS. Purpose of DAVIS. to visually explore the structure or pattern of data. Components of DAVIS.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

Data Exploration with DAVIS


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
    Presentation Transcript
    1. Moon HUH1, KwangRyeol SONG2, YoungSuk PARK1, KyungWook Shim 1Sungkyunkwan University, Seoul, Korea 2 Kwansei Research Institute, Seoul, Korea Data Exploration with DAVIS Variable Selection

    2. Purpose of DAVIS to visually explore the structure or pattern of data Variable Selection

    3. Components of DAVIS Data Manipulation Statistical Tools Plots Graphic Controllers Variable Selection

    4. Data Manipulation • Observation/variable selection • Focusing/deleting a subset of data set • Missing value process • Discretization Variable Selection

    5. Plots - Univariate • Bar Charts • Histogram • QQ Plot • FEDF • BoxPlot • Parallel Coordinates Variable Selection

    6. BoxPlot: Features • Standardization • Indentification Variable Selection

    7. Parallel Coordinates: Features • Direction of Plotting: Horizontal / Vertical • Ordering of the Variables: Component / Permutation • Jittering Variable Selection

    8. Parallel Coordinates -options Variable Selection

    9. Plots-multivariate • Scatterplot • Loess curve fitting • Touring • Dendrogram • Line Mosaic Plot • PCA plot Variable Selection

    10. Scatterplot-options Variable Selection

    11. Touring –GrandTour/Tracking Variable Selection

    12. Dendrogram –Agglomeration /Distance options Variable Selection

    13. Line Mosaic Plot –for discrete data Variable Selection

    14. PCA plot Variable Selection

    15. Real time grouping with DAVIS - hiliting • Manually grouping the data set into 2 subsets by mouse brushing a subset of data • Always can go back to the original data set Variable Selection

    16. Real time grouping with DAVIS–deleting/focusing Variable Selection

    17. Interactive Clustering with DAVIS-linking Variable Selection

    18. Clustering with DAVIS: EM with 3 groups Variable Selection

    19. Coloring a subset –outlier detection Variable Selection

    20. Touring with DAVIS- Tracking • Can investigate multidimensional structure of the data Variable Selection

    21. Data exploration with Decision Trees-Titanic data Variable Selection

    22. Decision Trees-2 Variable Selection

    23. Variable selection with DAVIS • Target (Class) variable discrete (nominal) type • Candidate variables nominal, numerical, and complex type Variable Selection

    24. Variable subset selection methods • MDI( Lee and Huh, 2003) . using p-values for the test statistics between the 2 variables. –log (p-value) is suggested • ReliefF (Kira and Randell, 1992) Relief (x)=P[different value of X | different class] -          P[different value of X | same class] • Mutual Information (originated by Shanon, 1948 and used for the measure of dependence by Perez, 1957, Russian) Darbellay (1999, CSDA) gives a good survey on the measure of statistical dependenceusing MI Variable Selection

    25. Subset selection with DAVIS– ranking variables • MDI(meaured of departure from indep.) • ReliefF • MI (measure of Information) Variable Selection

    26. Subset selection with DAVIS-decision trees • Discretization required Variable Selection

    27. Subset Selection with DAVIS- stepwise discriminant analysis • Continuous variables only • Good under normality Variable Selection

    28. Subset Selection with DAVIS- Mutual Information • Conventional approach: Discretization required • Normal mixture approach: Good for continuous variables • Incremental Algorith: Good for complex data Variable Selection

    29. Variable Selection with DAVIS-design layout Variable Selection

    30. Variable selection –titanic data • Variable ranking: sex, class, age • subset selection: age, class Variable Selection

    31. Concluding remarks • DAVIS is a Java-based system • Any statistical model can be added to the system as a visual component if it follows certain rules. • Need more efficient design layout for various strategies of variable selection. • Need to coin easier-to-understand terminologies for various elements of the component. Variable Selection