1 / 8

Foundational Issues

Machine Learning 726 Simon Fraser University. Foundational Issues. Outline. Functions vs. Probabilities The Curse of Dimensionality Bishop: Ch. 1. Learning Functions. Much learning is about predicting the value of a function from a list of features.

abla
Download Presentation

Foundational Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Machine Learning 726 Simon Fraser University Foundational Issues

  2. Outline • Functions vs. Probabilities • The Curse of Dimensionality • Bishop: Ch. 1.

  3. Learning Functions • Much learning is about predicting the value of a function from a list of features. • Classification: discrete function values. • Regression: continuous function values. • Mathematical Representation: map a feature vector x to a target value yi.e. f(x)=y. • Oliver’s heel pain example. • Often most intuitive to think in terms of function learning.

  4. Why Probabilities • Another view: the goal is learning the probability of an outcome. • Advantages: • Rank outcomes, quantify uncertainty. • Deals with noisy data. • Helps with combining predictions and pipelining. • Can incorporate base rate information (e.g., only 10% of heel pain is caused by tumor). • Can incorporate knowledge about inverse function, e.g., from diagnosis to symptom. • Bayes’ theorem: single formula with base rates and inverse probabilities.

  5. Why not probabilities • Disadvantage: exact numbers may be hard to get, more than needed.

  6. From Functions to Probabilities • Function + noise = probability. • See scatterplot, logistic regression.

  7. From Probabilities to Functions • Can model learning probability of y given x as function learning: f(x,y) = P(y|x). • E.g., neural nets for computing probabilities.

  8. The curse of dimensionality • In many applications, we have an abundance of features. • e.g., 20x20 image = 400 pixel values. • Scaling standard ML methods to high-dimensional feature spaces is hard, both computationally and statistically. • Statistics: data do not cover space. • Typically only few of the possible data settings occur. • manifold learning. • learning aggregate, global, or high-level features. • Unsupervised learning of feature hierarchies: deep learning. • Discussion Question: does the brain do deep learning?

More Related