1 / 22

Presented by: Lu He

Fast and Scalable Training of Semi-Supervised CRFs with Application to Activity Recognition

dillian
Download Presentation

Presented by: Lu He

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Fast and Scalable Training of Semi-Supervised CRFs with Application to Activity Recognition Maryam Mahdaviani, Tanzeem Choudhury Presented by: Lu He

  2. Challenge Collect labeled data is expensive in activity Recognition and also in many other domains

  3. Challenge Training complex CRF models with large numbers of input features is slow, and exact inference is often intractable. The ability to select the most informative features as needed can reduce the computational time and the risk of over-fitting of parameters.

  4. Solution A semi-supervised virtual evidence boosting (sVEB) algorithm for training Conditional Random Fields(CRFs) (Simultaneous Feature Selection and Parameter Learning)

  5. Conditional Random Fields Undirected graphical models Directly represent the conditional distribution over hidden states from observations, using clique potentials Make no assumption about dependency structure between observations (different from HMM)

  6. Inference & Parameter learning in CRFs Maximum likelihood Parameter Estimation: Problem: large CRFs exact inference is often intractable

  7. Inference & Parameter learning in CRFs Maximum pseudo-likelihood Estimation: Problem: In some cases, over-estimate the dependency parameters

  8. Boosting A kind of learning approach Sequentially learning a set of weak classifiers (weak learner) and combining them for final decisions Weak learner in VEB for training CRFs is a certain combination of features

  9. Virtual Evidence • vei={ve(xi), ve(n(yi)) } • a distribution over its domain rather than a single, observed value • Two types of observations nodes: • Hard evidence ve(xi): • local observation values • Soft evidence ve(n(yi)): • messages from neighbors

  10. Virtual Evidence Boosting Integrate boosting based feature selection into CRFs training Cutting a CRFs into individual patches (Like MPI), using these patches as training instances for boosting use the messages from the neighboring nodes as virtual evidence instead of using the true labels of neighbors, reducing over-estimation of neighborhood dependencies

  11. Virtual Evidence Boosting The objective function: Weighted least square error(WLSE) problem:

  12. Virtual Evidence Boosting

  13. Virtual Evidence Boosting • Feature Selection: • For a continuous attribute x(k), the weak learner is a linear combination of decision stumps: • Fora discrete attribute x(k) ∈ {1, · · ·,D}, the weak learner is:

  14. Virtual Evidence Boosting Feature Selection: 3. For a certain type of neighbor and corresponding virtual evidence vei(yk), the weak learner is the weighted sum of two indicator functions (compatibility features):

  15. Virtual Evidence Boosting Feature Selection: Unify the three cases for computing optimal feature weights: Cdi is the count of feature d in data instance i, can be 0 or 1 for local features and a real number between 0 and 1 for compatibility features

  16. Semi-supervised training Method: utilize unlabeled data via entropy regulation Strategy: Find labeling of the unlabeled data that mutually reinforces the supervised labels

  17. Semi-supervised Virtual Evidence boosting The objective function: Weighted least square error(WLSE) problem:

  18. Semi-supervised Virtual Evidence boosting

  19. Experiment Synthetic Data

  20. Experiment Activity dataset

  21. Strength and limitation?

  22. Question?

More Related