A Search-Classify Approach for Cluttered Indoor Scene Understanding
Download
1 / 24

A Search-Classify Approach for Cluttered Indoor Scene Understanding - PowerPoint PPT Presentation


  • 94 Views
  • Uploaded on

A Search-Classify Approach for Cluttered Indoor Scene Understanding. Liangliang Nan 1 , Ke Xie 1 , Andrei Sharf 2. 1 SIAT , China 2 Ben Gurion University, Israel . Digitalization of indoor scenes. Indoor scenes from Google 3D Warehouse. Acquisition of indoor scenes. Goal.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' A Search-Classify Approach for Cluttered Indoor Scene Understanding' - sally


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

A Search-Classify Approach for Cluttered Indoor Scene Understanding

Liangliang Nan1, Ke Xie1, Andrei Sharf2

1 SIAT, China

2 Ben Gurion University, Israel


Digitalization of indoor scenes Understanding

Indoor scenes from Google 3D Warehouse



Goal Understanding

  • Scene understanding


Challenges
Challenges Understanding

  • Clutter

    • Densely populated

    • Arbitrary arrangements

    • Partial representation

    • Occlusions

  • Complex geometry


Classification segmentation
Classification & Segmentation Understanding

  • Two interleaved problems

    • What are the objects?

    • Where are the objects?

  • Chicken-egg problem

    • Classification needs segmentation

    • Segmentation needs a prior


Our solution
Our solution Understanding

  • Search

    • Propagate / accumulate patches

  • Classify

    • Query classifier to detect object


Related work
Related Work Understanding

  • Indoor scenes (This Session)

  • [Fisher et al. 2012] [Shao et al. 2012] [Kim et al. 2012]

    • Semantic relationship

  • [Fisher et al. 2010, 2011]

  • Recognition using depth + texture (RGB-D)

    • [Quigley et al.2009], [Lai and Fox 2010]

      • Outdoor classification

    • [Golovinskiyet al. 2009]

      • Semantic labeling

    • [Koppulaet al. 2011]

      Controlled region growing process


  • Our search classify idea
    Our search-classify idea Understanding

    0.6

    0.8

    0.92

    0.94

    0.94

    0.94


    Method o verview
    Method Understandingoverview

    Training

    Search-Classify


    Point cloud features
    Point cloud features Understanding

    • Height-size ratio of BBox

    • Aspect ratio of each layer

    • Bottom-top, mid-top size ratio

    • Change in COM along horizontal slabs

    Bh

    Bd

    Bw


    Classifier
    Classifier Understanding

    • Handle missing data

      • Occlusion

    • Random decision forest

      • Efficient multi-class classifier

    • Trained with both scanned and synthetic data

      • Manually segmented and labeled

      • 510 chairs

      • 250 tables

      • 110 cabinets

      • 40 monitors etc.

    [Shottonet al. 2008, 2011]


    Search classify
    Search-Classify Understanding

    • Starts from seeds

      • Random patch triplets

      • Remove seeds with low confidence

    • Accumulating neighbor patches

      • Highest classification confidence

    • Stop condition

      • Steep decrease in classification confidence

    0.65

    0.92

    0.93

    0.88

    Seed


    Segmentation refinement by template fitting Understanding

    • Segmented - classified objects problems

      • Overlap, outliers, ambiguities etc.

    • Refinement

      • Outliers = patches with large distance


    Template deformation
    Template deformation Understanding

    • Different styles for each class

    • Predefined scalable parts

    • Templates can deform

    [Xuet al. 2010]


    Template deformation1
    Template deformation Understanding

    • Different styles for each class

    • Predefined scalable parts

    • Templates can deform

    [Xuet al. 2010]


    Fitting via template deformation
    Fitting Understandingvia template deformation

    • Best matching template

      • One-side Euclidean distance from points to template

    Confidence

    Fitting error

    Best fitting


    Results and discussion
    Results Understandingand discussion


    Results and discussion1
    Results Understandingand discussion


    Results and discussion2
    Results Understandingand discussion

    • Scalability test with varied object density

    0 (25) 1 (45) 5 (60)


    Results and discussion3
    Results Understandingand discussion

    Lai et al. 2011

    • Comparison

    Ours


    Limitation
    Limitation Understanding

    • Upward assumption

      • Features

      • Template fitting


    Future work
    Future work Understanding

    • Contextual information


    Thank you

    Thank you Understanding


    ad