1 / 19

S mart S inglesourcing with TURK

S mart S inglesourcing with TURK. Todd W. Schiller Kıvanç Muşlu **. PROBLEM. Crowdsourcing has a myriad of problems: Users must break down their problem into multiple simple tasks Reliable results require redundancy and complex workflows

glynn
Download Presentation

S mart S inglesourcing with TURK

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Smart Singlesourcing with TURK Todd W. Schiller KıvançMuşlu ** PoCSci 2011

  2. PROBLEM Crowdsourcing has a myriad of problems: • Users must break down their problem into multiple simple tasks • Reliable results require redundancy and complex workflows • Users don’t know if their task will be completed or not Introducing: … PoCSci 2011

  3. TURK: “SMART” SINGLESOURCING • TURK: Telemetry & computation Utilizing only effoRtbyKıvanç • Tasks are all performed by Kıvanç, a Turkish PhD student at the University of Washington mmmm… TURKEY PoCSci 2011

  4. LIVE DEMO PoCSci 2011

  5. WORKFLOW Send Request Receive Response The Kloud PoCSci 2011

  6. The TURK workforce PoCSci 2011

  7. GENDER PoCSci 2011

  8. SKILL & EDUCATION PoCSci 2011

  9. SERVICE AVAILABILITY PoCSci 2011

  10. NO VARIABLE COSTS GRADUATE STIPEND PoCSci 2011

  11. INELASTIC LABOR SUPPLY TURK consistently gives 110% Reservation Wage PoCSci 2011

  12. EXPERIMENTS PoCSci 2011

  13. Experiment #1: Doodles • Posted projects on TURK and Mechanical Turk to create a doodle: • Space Ship • Dinosaur PoCSci 2011

  14. Experiment #1: Results Mechanical Turk TURK • “What does doodle mean?” PoCSci 2011

  15. Experiment #1: Results (Continued) Mechanical Turk TURK PoCSci 2011

  16. Experiment #2: Original Research • Posted projects on TURK and Mechanical Turk to: • Find unknown bugs in the Java JDK using the Checker Framework • Co-author a software engineering conference paper PoCSci 2011

  17. Experiment #2: Results Mechanical Turk TURK • “Building and using pluggable type-checkers”by Werner Dietl, Stephanie Dietzel, Michael D. Ernst, KıvançMuşlu, and Todd Schiller. In ICSE'11, Proceedings of the 33rd International Conference on Software Engineering, (Waikiki, Hawaii, USA), May 25-27, 2011. PoCSci 2011

  18. Conclusions • TURK is a tool / marketplace for singlesourcing work • TURK is not a sufficient replacement for on-demand doodling technologies • TURK can perform software engineering research tasks • Additionally, TURK has numerous other advantages over crowdsourcing: • Perfect wage inelasticity • No variable costs • TURK will sometime bring Turkish Delight to SE group meetings PoCSci 2011

  19. Backup Slides PoCSci 2011

More Related