1 / 64

Human Computation - PowerPoint PPT Presentation

  • Uploaded on

Human Computation. Luis von Ahn. Carnegie Mellon University. Verification technology developed in collaboration with Carnegie Mellon University. “. Help! I’ve been waiting for 20 minutes, and nothing happens. ”.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Human Computation' - iorwen

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript


Luis von Ahn

Carnegie Mellon University

Help! I’ve been waiting for 20 minutes, and nothing happens

Verification technology developed in collaboration with Carnegie Mellon University


Which is the best Computer Science Grad School in the US?









CAPTCHA Sweat Shops

Spam companies hire humans to type CAPTCHAs all day long

$2.50 per hour for each human

720 captchas per hour per human

1/3 cent per account

Free Nude Photos

Type the word in the box if you want to see the next picture


Roughly 200 million CAPTCHAs are typed every day

10s of human time per CAPTCHA

Can we use this human effort for good?

Digitizing Books



DE xi Ii ARIi..

COPBCtrAGE\, DeC. I l 1 QOn.

Tlie royal palace at Frieclericheberg took fire last night. The himself directed the operations fns extinguishing the lire. The whole structure ,, ed.

Word that OCR can’t read

Scanned book

Randomly distorted

image of word

Two Words Instead of One?

Typing two English words takes as long as typing 6-8 random characters

Where do the Words Come From?

Years 1851-1980 will be completed in 2009


(6% of the world’s population)

CAPTCHA Sweat Shops

Mahiddin: Would you help us find some Captcha Project?

reCAPTCHA Support: What kind of CAPTCHA project are you looking for?

Mahiddin: We are manually break different type of CAPTCHA right now. We have the ability type every day many Captcha. Tell us how we can help you.

reCAPTCHA Support: How fast can you type?

Mahiddin: Personally I can type more than 60 WPM. But I have a big team. We can give you lot of product.

reCAPTCHA Support: Do you mind helping spammers?

Mahiddin: No problem, you can tell me details first.

reCAPTCHA Support: One last question: how do you deal with IP address blocking?

Please type the following paragraph:

Reusing Wasted Cycles

Solving CAPTCHAs


Bathroom (17m)

Electronic Games (1.1h)

Eating and Drinking (1.1h)

Household Chores (1h)

Sleeping (7.6h)

Caring for Others (1.1h)

Email and IM (1.7h)

TV (2.4h)

Other (7.6h)

9 Billion Human-Hours of Solitaire were Played in 2003

Empire State Building

7 million human-hours

(6.8 hours of solitaire)

Panama Canal

20 million human-hours

(less than a day of solitaire)

Labeling Images With Words




Super Evil

Still an open problem

Image Search On The Web

Uses filenames and html text


A method that can label all images on the web

Fast and cheap

Using Humans


The ESP game could label all images on google in a few weeks

The ESP game

The ESP Game

Two-player online game

Partners don’t know each other and can’t communicate

Object of the game:

Type the same word

Only thing in common is

An image

The ESP Game is Fun

50 million agreements with 200,000 players

There are many people that play over 20 hours a week

Labeling The Entire Web

5000 people playing simultaneously can label all images on Google in 2 months!

Individual games in Yahoo! and MSN average over 5,000 players at a time

Guesses Over Time

younger than 40


a male




shaved head

Sample Labels

Coming soon:

Meet your soul mate through the game!



mr. wilson










The esp game gives its players a weird and beautiful sense of anonymous intimacy. On the one hand, you have no idea who your partner is. On the other hand, the two of you are bringing your minds together in a way that lovers would envy.

The ESP game gives its players a weird and beautiful sense of anonymous intimacy.

The esp game gives its players a weird and beautiful sense of anonymous int On the one hand, you have no idea who your partner is.

It’s fast-paced

It’s like crack!

Helps me learn English

Strangely addictive

It’s so much fun trying to guess what others think. You have to step outside of yourself to match

In other crowning achievements, I haven't played in at least a week, and I am still on the top ten list of players on the ESP game. Number eight if I remember correctly.

God, my life is sad.


But by far the most intriguing aspect of the game is how often your random, unknown partner is a complete idiot.

Many fascinating things about the ESP Game:

How it uses a game structure to build an index of images on the web

What words become taboo from frequent use

How interesting some of the images pulled randomlyfrom the web are

Dear Luis von Ahn,

Your game is a fraud

I do research on ESP (extra sensory perception), and have an online ESP telepathy test on my website

Please refrain from using the term ESP, as it makes us legitimate ESP researchers look like a fraud

Audio ESP


Two-Player Game

Both players can see each other’s guesses

Players may or may not be listening to the same song

Object of the game:

Tell if you’re both hearing the same thing

Edith Law


loud electric guitar

electric guitar


male singer





Falls Pakistans Geschichte ein Indikator ist, so könnte Musharrafs Entscheidung, das Kriegsrecht zu verhängen, jener sprichwörtliche Tropfen sein, der das Fass zum Überlaufen bringt.


If Pakistan's history is any indicator, Musharraf's decision to impose martial law may prove to be the proverbial straw that breaks the camel's back.


Our Method

If Pakistan's history is an indicator, Musharraf's decision to impose martial law could be the straw that breaks the camel's back.

Slide Design

Luis von Ahn


Luis von Ahn

Manuel Blum

Nicholas Hopper

John Langford

The ESP Game Crew

Luis von Ahn

Laura Dabbish

David Stork

Latanya Sweeney

Jenn Tam

Anthony Tomasic

Shingo Uchihashi

Manuela Veloso

Jeannette Wing

Wardrobe Provided By

Charles Spiegel

This Work Has Been Partially Supported By


Microsoft Research

Google, Inc

The MacArthur Foundation

Susan Hrishenko

Takeo Kanade

Mihir Kedia

John Langford

Edith Law

Bryant Lee

Peter Lee

Roy Liu

Jitendra Malik

Udi Manber

Ben Maurer

Colin McMillen

Moni Naor

Lenore Ramm

Roni Rosenfeld

Push Singh

Special Thanks To

Aditya Akella

Josh Benaloh

Lenore Blum

Manuel Blum

Randy Bryant

Sharon Burks

Shuchi Chawla

Mike Crawford

Laura Dabbish

Alexei Efros

Alan Eustace

Bill Gates

Shiry Ginosar

Severin Hacker

Nick Hopper


Luis von Ahn

Ben Maurer

Colin McMillen

David Abraham

Mike Crawford

Tag a Tune Crew

Edith Law

Luis von Ahn

Roger Dannenberg

Speaker’s Entertainment

Provided By