1 / 5

Crowdsourcing Ontology Engineering: Enhancing Efficiency and Accuracy

Explore the world of crowdsourcing for ontology engineering, including collaborative approaches, challenges, experiments using MTurk and CrowdFlower, and open questions relating to quality assurance, incentives, and crowdsourcing approaches.

theola
Download Presentation

Crowdsourcing Ontology Engineering: Enhancing Efficiency and Accuracy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Crowdsourcing ontology engineering Elena Simperl Web and Internet Science, University of Southampton 11 April 2013

  2. Overview • "online, distributed problem-solving and production model“ [Brabham, 2008] • Varieties: wisdom of the crowds/collective intelligence, open innovation, human computation... • Why is it a good idea? • Cost and efficiency savings • Wider acceptance, closer to user needs, diversity • Approaches • Collaborative ontology engineering • Challenges/competitions • Games with a purpose • Microtask/paid crowdsourcing • In combination with automatic techniques

  3. Crowdsourcing ontology alignment • Experiments usingMTurk, CrowdFlowerandestablishedbenchmarks • Enhancingtheresultsofautomatictechniques • Fast, accurate, cost-effective [Sarasua, Simperl, Noy, ISWC2012]

  4. Open questions • Quality assurance and evaluation • Incentives and motivators • Choice of crowdsourcing approach and combinations of different approaches • Reusable collection of algorithms for quality assurance, task assignment, workflow management, results consolidation etc • Schemas recording provenance of crowdsourced data • Descriptive framework for classification of human computation systems • Typesoftasksandtheirmodeofexecution • Participantsandtheirroles • Interaction withsystemandamongparticipants • Validation ofresults • Consolidationandaggregationofinputsintocompletesolution

  5. Theory and practice of social machineswww.sociam.org http://sociam.org/www2013/

More Related