Object recognition based on shape and function
Download
1 / 34

Object Recognition based on Shape and Function - PowerPoint PPT Presentation


  • 146 Views
  • Uploaded on

Object Recognition based on Shape and Function. Akihiro Eguchi BS Thesis Defense – November 30, 2011. Committee Craig Thompson Russell Deaton John Gauch. College of Engineering Computer Science and Computer Engineering Department. Outline. Personal Background Introduction Background

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Object Recognition based on Shape and Function ' - bennett-delaney


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Object recognition based on shape and function

Object Recognitionbased on Shape and Function

Akihiro Eguchi

BS Thesis Defense – November 30, 2011

Committee

Craig Thompson

Russell Deaton

John Gauch

College of Engineering

Computer Science and Computer Engineering Department


Outline
Outline

  • Personal Background

  • Introduction

  • Background

  • Approach and Architecture

  • Methodology, Results, and Analysis

  • Conclusion


1 personal background

2. Introduction

3. Background

4. Approach and Architecture

5. Methodology, Results, and Analysis

6. Conclusion

1.Personal background


Akihiro eguchi
Akihiro Eguchi

  • Majors at the University of Arkansas

    • College of Engineering

      • Honors B.S., Computer Science

        • Advisor: Dr. Craig Thompson

    • Fulbright College of Fine Arts and Science

      • Honors B.A., Psychology

        • Thesis: “Cultural bias during word learning”

        • Advisor: Dr. Douglas Behrend

      • Minor in B.S., Mathematics


Projects publications awards
Projects, Publications, Awards

  • Projects in Computer Science field

    • Semantic World

      • Publications:

        • International Journal of Computer Information Systems and Industrial Management (2011)

        • Web Virtual Reality and Three-Dimensional Worlds Workshop, Freiburg, Germany (2010)

        • Inquiry Journal of Undergraduate Research (2010)

        • Cyber Infrastructure Days Conference (2010)

        • Conference on Applied Research in Information Technology (2010)

        • X10 Workshop on Extensible Virtual Worlds (2010)

      • Awards:

        • Honorable Mention, CRA Outstanding Undergraduate Researchers Award (2011)

        • Winner, University of Arkansas Undergraduate Research Award, U of A. (2010)

    • Autonomous Floor Mapping Robot

      • Publications:

        • 9th IEEE International Symposium on Robotic and Sensors Environments (ROSE), Montreal, QC, Canada (2011)

    • Minority Game

      • Publications:

        • 5th IEEE International Workshop on Multi-Agent Systems and Simulation (MAS&S), Szczecin, Poland (2011)

    • Smart Housekeeper Robot with Android

  • Projects in Psychology field

    • Cultural bias during children’s word learning

    • Cultural differences in identity perspectives


2 introduction

  • 1. Personal Background

    3. Background

    4. Approach and Architecture

    5. Methodology, Results, and Analysis

    6. Conclusion

2. Introduction


Earlier object recognition
Earlier Object Recognition

  • Required

    • extensive knowledge of math

    • expensive equipment

      use of Kinect

  • Mostly based only on shape of objects

    • difficult to recognize objects like

      • a uniquely designed chair

      • different objects that have similar shapes

        ideas from Developmental Psychology

        “Function Bias”


Objective
Objective

  • To combine Kinect sensor with machine learning techniques

  • To develop a new object recognition model based on shape bias and function bias


3 background

  • 1. Personal Background

  • 2. Introduction

    4. Approach and Architecture

    5. Methodology, Results, and Analysis

    6. Conclusion

3. Background


Key concept 1 machine learning
Key Concept 1Machine Learning

  • Give computers a way to learn without explicitly being programmed

    • autonomous vehicles, checker playing, and signal processing, etc.

  • Predict the function f(x) = y

  • Examples include

    • rote learning

    • decision tree based (ID3)

    • neural networks

    • k-nearest neighbor clustering


Key concept 2 microsoft kinect for the xbox
Key Concept 2Microsoft Kinect for the Xbox

  • A controller for the Microsoft gaming console Xbox 360 costs only $150

    • dynamic depth image retrieval

    • human body recognition

    • skeletal joint tracking

    • multi-array microphone

  • Kinect SDK was officially released in June 2011


Related work developmental psychology
Related WorkDevelopmental Psychology

  • Human children use two main biases when learning to name objects:

    • Shape bias (B. Landau et al., 1988)

      • generalize name of the object if the shape is similar

    • Function bias (D. G. K. Nelson et al., 2000)

      • generalize name of the object if the function of the use seem to be the same

  • B. Landau, L. Smith, and S. Jones, “The Importance of Shape in Early Lexical Learning,” Cognitive Development, vol. 3, no. 3, 1988, pp. 299-321.

  • D. G. K. Nelson, R. Russell, N. Duke, and K. Jones, “Two-Year-Olds Will Name Artifacts by Their Functions,” Child Development, vol. 71, no. 5, 2000, pp. 1271-1288.


Related work object recognition in cs
Related WorkObject Recognition in CS

  • Simulation of biases (Grabner et al., 2011)

    • Shape bias:

      • Define 3D models that are labeled with a same name (e.g., chair) and run a machine learning algorithm to train the classifier.

    • Function bias:

      • First define the use of the object; e.g., chair is a object to sit on.

      • Then, let the program learn the posture of sitting.

      • If the object is sittable, the objectis more likely to be a chair.

  • H. Grabner, J. Gall, L. V. Gool, “What Makes a Chair a Chair?,” IEEE Conference on Computer Vision and Pattern Recognition (CVPR'11), Colorado Springs, CO, June 20-25, 2011, pp. 1529-1536.


Related work activity recognition
Related WorkActivity Recognition

  • Workflow analysis using RFID (Philipose et al., 2004)

    • Label objects with RFID to record action

    • Bayesian networks to infer actions

  • Video analysis with SVM (Schuldt et al., 2004)

    • 4 second video recording of walking, jogging, and running

    • SVM to train the classifier to recognize those actions.

  • M. Philipose, K. P. Fishkin, M. Perkowitz, D.J. Patterson, D. Fox, H. Kautz, D. Hahnel,, “Inferring Activities from Interactions with Objects,” IEEE Pervasive Computing, vol. 3, no. 4, Oct.-Dec. 2004, pp. 50-57.

  • C. Schuldt, I. Laptev, B. Caputo, “Recognizing Human Actions: a Local SVM Approach, ” Proceedings of the 17th International Conference on Pattern Recognition (ICPR), 2004.


4 approach and architecture

4. Approach and Architecture


Selecting a learning technique
Selecting a Learning Technique

  • hand-written number recognition

    • 7x11 grid of 77 cells

    • Different Techniques

      • Neural network (RBFN)

        • 5 nodes, 3 nodes, 300 iterations

        • accurate, but slow

      • K-NN clustering

        • simple and fast for small input.

        • less accurate but sufficient for this project


Learning to use the kinect sdk
Learning to Use the Kinect SDK

  • Noise reduction

  • 3D reconfiguration with OpenGL



Plain surface removal with ransac algorithm
Plain Surface Removal with RANSAC Algorithm

  • Random Sample Consensus (RANSAC)

    • randomly takes 3 points from the point cloud to determine a random plain

    • counts how many points are on the plain

    • iterating through to find a plain that has maximum number of points


K nn for shape learning
K-NN for Shape Learning

  • Learning:

    • User names each object or chooses the name from a list of names the user has already told the program

  • Testing

    • program compares the shape of the target object with previously learned shape information to infer the name


Activity recognition using the kinect
Activity Recognition using the Kinect

  • Learning:

    • Kinect tracks twenty different joints of skeletal information

    • records coordinates for each joint every 0.1 seconds for 10 seconds

    • name the activity

  • Testing:

    • Use of K-NN to infer the target activity


Demo activity recognition using the kinect
Demo: Activity Recognition using the Kinect

  • http://www.youtube.com/watch?v=AxCn0eKWkiQ


Object recognition model with shape bias and function bias
Object Recognition Model with Shape Bias and Function Bias

  • Learning:

    • Let program learn the shape

    • Activity learning to learn the use

    • Associate the activity with the name of object.

  • Testing:

    • Based on the confidence:

      • "Maybe the object is [answer based on the shape]. But the object may be a [answer based on the action] because you used the object for [the name of action]"

      • "I think the object is [answer based on the action] because you used the object for [name of action]. But it might be a [answer based on the shape] based on the shape."


Demo object recognition model with shape bias and function bias
Demo: Object Recognition Model with Shape Bias and Function Bias

  • http://www.youtube.com/watch?v=4ia76fzxm68


5 methodology results and analysis

5. Methodology, Results, and Analysis


Methodology
Methodology Bias

  • Target Objects

    • two objects that look similar but have a different use

    • two other objects that look different but have the same name and function

a can of antiperspirant and a can of insecticide

a conventional chair and an oddly shape of a chair


Object recognition based only on shape
Object Recognition based only on Shape Bias

  • Learning:

    • Antiperspirant

    • Insecticide

    • Chair (1)

  • Testing:

    • Antiperspirant -> “Antiperspirant” / “Insecticide”

    • Insecticide -> “Antiperspirant” / “Insecticide”

    • Chair (1) -> “Chair”

    • Chair (2) -> “Antiperspirant” ???

      • Because shape is quite different from the chair (1)


Function recognition
Function Recognition Bias

  • Learning:

    • “killing bugs” with insecticide

    • “deodorizing” with antiperspirant

    • “sitting” on a chair (1)

  • Testing

    • Killing bugs with insecticide  “killing bugs”

    • Deodorizing with antiperspirant  “deodorizing”

    • Sitting on a chair (1)  “sitting”

    • Sitting on a chair (2)  “sitting”


Object recognition based on both shape and function
Object Recognition based on both Shape and Function Bias

  • “Deodorizing” with antiperspirant

     correctly answer OR

     "Maybe the object is insecticide. But the object may be antiperspirant because you used the object for deodorizing".

  • “Killing bugs” with insecticide

     correctly answer OR

     "Maybe the object is antiperspirant. But the object may be insecticide because you used the object for killing bugs".

  • Sitting on a Chair (1)

     correctly answer “chair”

  • Sitting on a Chair (2)

    "I think the object is a chair because you used the object for sitting. But it might be a antiperspirant based on the shape."


Analysis
Analysis Bias

  • when action does not involve a lot of movement, like sitting, the program works well

  • when action involves movement like walking, the shifting of timing can be a problem.


6 conclusion

  • 1. Personal Background Bias

  • 2. Introduction

    3. Background

    4. Approach and Architecture

    5. Methodology, Results, and Analysis

6. Conclusion


Summary
Summary Bias

  • Proposed a new way of designing a computational object recognition model

    • knowledge from developmental psychology

    • shape + function

  • Leveraged powerful features of the Kinect sensor

    • depth map retrieval

    • human body joint recognition

    • easier to program an advanced application

    • less expensive

  • Result shows that the model works as expected


Future work
Future Work Bias

  • The machine learning technique used for this model can be improved

  • Speech recognition technology can be used to name objects and actions

  • Can recognize sequences of activities (workflows)

    • brushing your teeth involves turning on the faucet, picking up the toothpaste, brushing, rinsing …

  • Help to create a future semantic world with smart objects

    • visual object recognition complements RFID tag based recognition

    • the accuracy can be improved by combining with ontology field of study


Questions
Questions Bias


ad