LUIS VON AHN
This presentation is the property of its rightful owner.
Sponsored Links
1 / 73

CAPTCHA, THE ESP GAME, AND OTHER STUFF PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on
  • Presentation posted in: General

LUIS VON AHN. CAPTCHA, THE ESP GAME, AND OTHER STUFF. CARNEGIE MELLON UNIVERSITY. CAPTCHA. A PROGRAM THAT CAN TELL WHETHER ITS USER IS A HUMAN OR A COMPUTER. CAPTCHA. A PROGRAM THAT CAN GENERATE AND GRADE TESTS THAT:. A.MOST HUMANS CAN PASS B.CURRENT COMPUTER PROGRAMS CANNOT PASS.

Download Presentation

CAPTCHA, THE ESP GAME, AND OTHER STUFF

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Captcha the esp game and other stuff

LUIS VON AHN

CAPTCHA,

THE ESP GAME, AND OTHER STUFF

CARNEGIE MELLON UNIVERSITY


Captcha the esp game and other stuff

CAPTCHA

A PROGRAM THAT CAN TELL WHETHER ITS USER IS A HUMAN OR A COMPUTER


Captcha the esp game and other stuff

CAPTCHA

A PROGRAM THAT CAN GENERATE AND GRADE TESTS THAT:

A.MOST HUMANS CAN PASS

B.CURRENT COMPUTER PROGRAMS CANNOT PASS


Captcha the esp game and other stuff

PICKS RANDOM STRING OF LETTERS

RENDERS IT INTOA DISTORTED IMAGE

oamg

EXAMPLE


Captcha the esp game and other stuff

…AND GENERATES A TEST:

TYPE THE CHARACTERS THAT APPEAR IN THE IMAGE


Captcha the esp game and other stuff

P STANDS FOR PUBLIC

ALL CODE AND DATA USED BY A CAPTCHA SHOULD BE PUBLICLY AVAILABLE


Captcha the esp game and other stuff

COMPLETELY AUTOMATED PUBLIC TURING TEST TO TELL COMPUTERS AND HUMANS APART


Captcha the esp game and other stuff

UTILIZING THE POWER OF HUMAN CYCLES

CAPTCHA

APPLICATIONS

EXAMPLES

DIRTY HACKS

THE ESP GAME

OTHER STUFF


Captcha the esp game and other stuff

APPLICATIONS OF CAPTCHAS


Captcha the esp game and other stuff

QUICKVOTE

Which is the best Computer Science Grad School in the US?

Berkeley

MIT

CMU

Princeton

Cornell

Stanford

vote

(FROM WWW.SLASHDOT.ORG)


Captcha the esp game and other stuff

APPLICATIONS

FREE E-MAIL SERVICES

DATA COLLECTION

WORMS AND SPAM

PREVENTING DICTIONARY ATTACKS (PINKAS AND SANDER ‘02)


Captcha the esp game and other stuff

APPLICATIONS

FREE E-MAIL SERVICES

DATA COLLECTION

WORMS AND SPAM

PREVENTING DICTIONARY ATTACKS (PINKAS AND SANDER ‘02)


Captcha the esp game and other stuff

APPLICATIONS

FREE E-MAIL SERVICES

DATA COLLECTION

WORMS AND SPAM

PREVENTING DICTIONARY ATTACKS (PINKAS AND SANDER ‘02)


Captcha the esp game and other stuff

APPLICATIONS

FREE E-MAIL SERVICES

DATA COLLECTION

WORMS AND SPAM

PREVENTING DICTIONARY ATTACKS (PINKAS AND SANDER ‘02)


Captcha the esp game and other stuff

EXAMPLES OF CAPTCHAS


Captcha the esp game and other stuff

PIX

WHAT ARE THESE PICTURES OF?


Captcha the esp game and other stuff

PIX

WHAT ARE THESE PICTURES OF?

THE IMAGES NEED TO BE RANDOMLY DISTORTED


Captcha the esp game and other stuff

PIX

WHAT ARE THESE PICTURES OF?

THE IMAGES NEED TO BE RANDOMLY DISTORTED


Captcha the esp game and other stuff

PIX

WHAT ARE THESE PICTURES OF?

THE IMAGES NEED TO BE RANDOMLY DISTORTED


Captcha the esp game and other stuff

WHAT ARE THESE PICTURES OF?


Captcha the esp game and other stuff

BONGO

VISUAL ANALOGY

PROBLEMS

(BONGARD 1951)


Captcha the esp game and other stuff

SOUND-BASED CAPTCHA

HUMANS ARE BETTER THAN COMPUTERS AT UNDERSTANDING SPOKEN LANGUAGE


Captcha the esp game and other stuff

OPEN PROBLEM

CREATE A CAPTCHA BASED ON LANGUAGE UNDERSTANDING


Captcha the esp game and other stuff

DIRTY HACKS


Captcha the esp game and other stuff

CAPTCHA SWEAT SHOPS

SPAM COMPANIES HIRE HUMANS TO SOLVE CAPTCHAS ALL DAY LONG

$2.50 PER HOUR FOR EACH HUMAN

720 CAPTCHAS PER HOUR PER HUMAN

1/3 CENT PER ACCOUNT


Captcha the esp game and other stuff

Free Nude Photos

> NEXT


Captcha the esp game and other stuff

Free Nude Photos

Type the word in the box if you want to see the next picture


Captcha the esp game and other stuff

UTILIZING THE POWER OF HUMAN CYCLES

CAPTCHA

APPLICATIONS

EXAMPLES

DIRTY HACKS

THE ESP GAME

OTHER STUFF


Captcha the esp game and other stuff

MARTHA STEWART

FLOWERS

SUPER EVIL

LABELING IMAGES WITH WORDS

STILL AN OPEN PROBLEM


Captcha the esp game and other stuff

USES FILENAMES AND HTML TEXT

IMAGE SEARCH ON THE WEB


Captcha the esp game and other stuff

REASON:

MOST IMAGES DON’T HAVE PROPER CAPTIONS

ACCESSIBILITY

LESS THAN 10% OF THE WEB IS ACCESSIBLE TO THE VISUALLY IMPAIRED


Captcha the esp game and other stuff

DESIDERATA

A METHOD THAT CAN LABEL ALL IMAGES ON THE WEB

FAST AND CHEAP


Captcha the esp game and other stuff

USING HUMANS

CLEVERLY

THE ESP GAME COULD LABEL ALL IMAGES ON THE WEB IN 30 DAYS!

THE ESP GAME


Captcha the esp game and other stuff

THE ESP GAME

TWO-PLAYER ONLINE GAME

PARTNERS DON’T KNOW EACH OTHER AND CAN’T COMMUNICATE

OBJECT OF THE GAME:

TYPE THE SAME WORD

THE ONLY THING IN COMMON IS AN IMAGE


Captcha the esp game and other stuff

GUESSING: CAR

GUESSING: BOY

GUESSING: CAR

GUESSING: HAT

GUESSING: KID

SUCCESS!

YOU AGREE ON CAR

SUCCESS!

YOU AGREE ON CAR

THE ESP GAME

PLAYER 1

PLAYER 2


Captcha the esp game and other stuff

© 2004 Carnegie Mellon University, all rights reserved. Patent Pending.


Captcha the esp game and other stuff

THE ESP GAME IS FUN

4.1 MILLION LABELS WITH 23,000 PLAYERS

THERE ARE MANY PEOPLE THAT PLAY OVER 20 HOURS A WEEK


Captcha the esp game and other stuff

LABELING THE ENTIRE WEB

5000 PEOPLE PLAYING SIMULTANEOUSLY CAN LABEL ALL IMAGES ON GOOGLE IN 30 DAYS!

INDIVIDUAL GAMES IN YAHOO! AND MSN AVERAGE OVER 5,000 PLAYERS AT A TIME


Captcha the esp game and other stuff

A FEW MILLION LABELS

CAN IMPROVE IMAGE SEARCH

CAN BE USED TO IMPROVE COMPUTER VISION


Captcha the esp game and other stuff

WE EMULATE PARTNER BY PLAYING PRE- RECORDED MOVES

SINGLE PLAYER GAME

A SINGLE PERSON CAN PLAY WITH PRE-RECORDED ACTIONS AS THEIR PARTNER

(0:12) CAR

(0:15) HAT

(0:21) KID

(0:08) BOY

(0:23) CAR

NOTICE THAT THIS DOESN’T STOP THE LABELING PROCESS!

WHEN 2 PEOPLE PLAY, WE RECORD EVERY ACTION WITH TIMING INFORMATION


Captcha the esp game and other stuff

WHAT ABOUT CHEATING?

IF A PAIR PLAYS TOO FAST, WE DON’T RECORD THE WORDS THEY AGREE ON


Captcha the esp game and other stuff

WE GIVE PLAYERS TEST IMAGES FOR WHICH WE KNOW ALL THE COMMON LABELS:

WE ONLY STORE A PLAYER’S GUESSES IF THEY SUCCESSFULLY LABEL THE TEST IMAGES

WHAT ABOUT CHEATING?


Captcha the esp game and other stuff

SEARCH RESULTS


Captcha the esp game and other stuff

BEACHCHAIRSSEAPEOPLEMAN

WOMANPLANTOCEAN

TALKINGWATERPORCH

SAMPLE LABELS


Captcha the esp game and other stuff

SADDAM

MR. WILSON

MAN

FACE

MOUSTACHE

COMING SOON:

MEET YOUR SOUL MATE THROUGH THE GAME!

BUSH

PRESIDENT

DUMB

YUCK

YUCK

SAMPLE LABELS


Captcha the esp game and other stuff

WHY DO PEOPLE LIKE THE ESP GAME?


Captcha the esp game and other stuff

THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY. ON THE ONE HAND, YOU HAVE NO IDEA WHO YOUR PARTNER IS. ON THE OTHER HAND, THE TWO OF YOU ARE BRINGING YOUR MINDS TOGETHER IN A WAY THAT LOVERS WOULD ENVY.

THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY.

THE ESP GAME GIVES ITS PLAYERS A WEIRD AND BEAUTIFUL SENSE OF ANONYMOUS INTIMACY. ON THE ONE HAND, YOU HAVE NO IDEA WHO YOUR PARTNER IS.


Captcha the esp game and other stuff

IT’S FAST-PACED

HELPS ME LEARN ENGLISH

STRANGELY ADDICTIVE

IT’S SO MUCH FUN TRYNG TO GUESS WHAT OTHERS THINK. YOU HAVE TO STEP OUTSIDE OF YOURSELF TO MATCH


Captcha the esp game and other stuff

IN OTHER CROWNING ACHEIVEMENTS,

I HAVEN'T PLAYED IN AT LEAST A WEEK (PROBABLY LONGER), AND I AM STILL ON THE TOP TEN LIST OF PLAYERS ON THE ESP GAME. NUMBER EIGHT IF I REMEMBER CORRECTLY.

GOD, MY LIFE IS SAD.

BLOGS


Captcha the esp game and other stuff

BUT BY FAR THE MOST INTRIGUING ASPECT OF THE GAME IS HOW OFTEN YOUR RANDOM, UNKNOWN PARTNER IS A COMPLETE IDIOT.

THERE ARE MANY FASCINATING THINGS ABOUT THE ESP GAME:

HOW IT USES A GAME STRUCTURE TO BUILD AN INDEX OF IMAGES ON THE WEB

WHAT WORDS BECOME TABOO FROM FREQUENT USE

HOW INTERESTING SOME OF THE IMAGES PULLED RANDOMLY FROM THE WEB ARE


Captcha the esp game and other stuff

OTHER GAMES


Captcha the esp game and other stuff

LOCATING OBJECTS IN IMAGES

THE ESP GAME TELLS US IF AN IMAGE CONTAINS A SPECIFIC OBJECT, BUT DOESN’T SAY WHERE IN THE IMAGE THE OBJECT IS

SUCH INFORMATION WOULD BE EXTREMELY USEFUL FOR COMPUTER VISION RESEARCH


Captcha the esp game and other stuff

PAINTBALL GAME

PLAYERS SHOOT AT OBJECTS ON THE IMAGE

SHOOT THE:

CAR

WE GIVE POINTS AND CHECK ACCURACY BY GIVING PLAYERS IMAGES FOR WHICH WE ALREADY KNOW WHERE THE OBJECT IS


Captcha the esp game and other stuff

PAINTBALL GAME

X% OF IMAGES

(100-X)% OF IMAGES

DON’T

KNOW


Captcha the esp game and other stuff

BALL


Captcha the esp game and other stuff

REVEALING IMAGES

GUESSER

REVEALER

CAR

BRUSH

CAR

BRUSH

CAR

GUESS

PARTNER’S

GUESS


Captcha the esp game and other stuff

MEDIUM-TERM GOAL

USE THE ESP GAME, ALONG WITH THE PAINT-BALL GAME AND THE REVEALING GAME TO CREATE THE MOST ACCURATE COMPUTER VISION ALGORITHMS TO RECOGNIZE A WIDE VARIETY OF OBJECT CATEGORIES


Captcha the esp game and other stuff

WE GIVE PEOPLE

40% OF IMAGES WITH PROPERTY X

40% OF IMAGES WITHOUT PROPERTY X

20% OF UNKNOWN IMAGES

X OR NOT

CAN BE USED TO CLASSIFY ALL IMAGES ON THE WEB AS HAVING PROPERTY X OR NOT

$$$

IF PEOPLE LIKE IMAGES WITH PROPERTY X, WE CAN OFFER AN INCENTIVE: THE MORE ACCURATE A PLAYER IS, THE MORE IMAGES WITH PROPERTY X THEY GET


Captcha the esp game and other stuff

THE POWER OF HUMAN CYCLES

CAPTCHA

OPEN

PROBLEM

GAME


Captcha the esp game and other stuff

OUR WORK ON CAPTCHA HAS APPEARED IN OVER 100 NEWSPAPERS AROUND THE WORLD


Captcha the esp game and other stuff

THE ESP GAME HAS APPEARED IN OVER 50 NEWSPAPERS AROUND THE WORLD


Captcha the esp game and other stuff

Name That Image

Computers excel at sifting information, but they have trouble distinguishing a picture of a tree from one of a turtle.

So researchers at Carnegie Mellon University knew they needed human volunteers to successfully label millions of unmarked images on the Web. Luis von Ahn, a graduate student in computer science, had the inspired idea of turning the laborious process into a two-player contest called the ESP Game (espgame.org).

Each time you play, you are randomly paired with another anonymous player. You can't communicate with your partner, although you both see the same image. The goal is to guess the descriptive word your partner is typing for the image. Once you both type the same word, you see a new image.

Since October the game has attracted 15,000 registered players, and they have created more than 1.5 million labels (a label is a single descriptive word; most photos require multiple labels). The site draws from a database of 200,000 Web images; another 200,000 are to be added soon. Mr. von Ahn said that properly labeling Web images would allow for more efficient image searching, improve the screen readers used by the visually impaired and help users block inappropriate images.


Captcha the esp game and other stuff

WWW.CAPTCHA.NET


Captcha the esp game and other stuff

WWW.ESPGAME.ORG


Captcha the esp game and other stuff

SLIDE DESIGN

LUIS VON AHN

CAPTCHA CREW

LUIS VON AHN

MANUEL BLUM

NICHOLAS HOPPER

JOHN LANGFORD

THE ESP GAME CREW

LUIS VON AHN

LAURA DABBISH

SPEAKER’S ENTERTAINMENT PROVIDED BY

HOMESTARRUNNER.COM

WARDROBE PROVIDED BY

THE GAP

HAIRSTYLES

ARNAUD SOME-FRENCH-LAST-NAME

THIS WORK HAS BEEN

AND WILL BE PARTIALLY SUPPORTED BY

MANUEL BLUM

THE ALADDIN CENTER

MICROSOFT RESEARCH

MY MOM

YOUR TAX DOLLARS

MORE SPECIAL THANKS

UDI MANBER

MONI NAOR

LENORE RAMM

CHUCK ROSENBERG

RONI ROSENFELD

DAVID STORK

LATANYA SWEENEY

SHINGO UCHIHASHI

MANUELA VELOSO

SPECIAL THANKS TO

ADITYA AKELLA

JOSH BENALOH

LENORE BLUM

MANUEL BLUM

ANDREW BORTZ

SHUCHI CHAWLA

LAURA DABBISH

BILL GATES

BRIGHTEN GODFREY

NICK HOPPER

SCOTT HUDSON

TAKEO KANADE

STEVEN KATZ

JOHN LANGFORD

JITENDRA MALIK


  • Login