1 / 8

Enhancing Computer Vision with Games for a Purpose

Explore using Games with a Purpose to improve Computer Vision tasks like image matching, face recognition, and sound tagging. Leverage human skills to enhance computer capabilities.

caldwell-whitfield
Download Presentation

Enhancing Computer Vision with Games for a Purpose

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ideas Session Willer Travassos, Jan. 24th

  2. GWAP • Games with a purpose (GWAP) uses the computational power of humans to perform tasks that computers are unable. • Up until now most of these games concentrate on simple tasks like describing music or picture. • These problems play on the fact that computers are horrible at reproducing the human senses, specially Vision. • So why not extend GWAP to solve/help to solve problems that are found in Computer Vision.

  3. Idea 1: Image Matching/Jigsaw Puzzle • Our eyes function as two cameras that keep constantly taking pictures of our surroundings. • Since we have two cameras in our head, we can assume that every time we look at a location we have two pictures of it, from different positions. • Our brain is then responsible to put together these two similar pictures

  4. Idea 1: Image Matching/Jigsaw Puzzle • Unfortunately, computers do not do a good job of reconstruction images like our brain. • They can, specially, miserably fail when we objects that are present one image and not in the other. Ex: occluded objects. • So the idea here is to break these images in several pieces, like in a jigsaw, so that humans can piece them together. • Also players can describe object positioning and tell us where certain objects are located or why they are not visible, and etc.

  5. Idea 2: Face Recognition/Guess Who? game • Computers can do face recognition with a certain degree of success, but they can be easily fooled in case contents of the picture changes. • Lighting, face angle, and several other factors in a picture may make a computer change, incorrectly, its decision to determine whether faces in two pictures match or not. • So why not use humans to describe, and determine if the people in a picture are the same

  6. Idea 2: Face Recognition/Guess Who? game • Remember back in your childhood. You may have come across a game that made you solve this problem in a fun manner. • The game in question is called “Guess Who?” • Instead of using cartoon images we could use real people images. • Images could be slightly different from each other so we can link different images together.

  7. Idea 3: Tagging sounds • GWAP mainly concentrate on Vision. Even though there is a game to tag songs, on the GWAP site, there is no game to describe/tag general sounds. • Ex: we may play sounds that may infer distance, fear, etc. • Tagging sounds directly does not seem as a fun activity, so maybe a new has to be developed. Any ideas?

  8. Resources • Luis von Ahn, and Laura Dabbish, “Labeling Images with a Computer Game”, ACM Conference on Human Factors in Computing Systems (CHI 2004), April 2004. • Luis von Ahn, Ruoran Liu and Manuel Blum, “Peekaboom: A Game for Locating Objects in Images”,ACM Conference on Human Factors in Computing Systems (CHI 2006), April 2006. • Luis von Ahn, Mihir Kediaand Manuel Blum, “Verbosity: A Game for Collecting Common-Sense Facts”, ACM Conference on Human Factors in Computing Systems (CHI 2006), April 2006.

More Related