The assembly line for the information age
This presentation is the property of its rightful owner.
Sponsored Links
1 / 20

The “ Assembly Line ” for the Information Age PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on
  • Presentation posted in: General

The “ Assembly Line ” for the Information Age. Human-Computer Cooperation for Large-Scale Product Classification. Jianfu Chen Computer Science Department, Stony Brook University. Machines Transform Human History.

Download Presentation

The “ Assembly Line ” for the Information Age

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


The assembly line for the information age

The “Assembly Line” for the Information Age

Human-Computer Cooperation for Large-Scale Product Classification

Jianfu Chen

Computer Science Department,

Stony Brook University


Machines transform human history

Machines Transform Human History


People have always been seeking the optimal way of integrating machine and human labor

People have always been seeking the optimal way of integrating machine and human labor.


20 th century

20th Century

Ford Assembly Line

Integrates Machine and Human Labor Efficiently


21 st century information age

21st Century – Information Age

“Mass Production” of Information


We want to find the optimal ways to integrate machine and human intelligence

We want to find the optimal ways to integrate machine and human intelligence.

  • NOT all products could be produced fully automatically by machines

    • assembly line integrated machine & human labor

  • NOT all information can be produced fully automatically by computers

    • We want to find optimal ways to integrate machine and human intelligence

  • What’s the “Assembly Line” for the Information Age?


A case study large scale product classification

A Case Study: Large scale product classification

Kindle Fire HD 8.9" 4G LTE Wireless

8.9" HD Display, Dolby Audio, Dual-Band Dual-Antenna Wi-Fi, 4G LTE, 32GB or 64GB

  • Goal:

  • optimally integrate computer and human effort

  • Achieve a lower unit cost for product classification

  • More precisely, optimize the accuracy-cost tradeoff


An assembly line for human computer cooperation

An “Assembly Line” for Human Computer Cooperation

A list of K candidate classes

3Com V.35 cable

V.35 cable ( DTE ) - DB-50 (M) - M/34 (V.35) (M) - 10 ft

26121609

Machine Accuracy

Human Accuracy

System Accuracy

=

X

Costis Human labor cost, i.e., the salary paid to workers, which is proportional to the working time spent.


A quick glance at accuracy cost relation

A quick glance at Accuracy-Cost Relation

  • Assume K determines the Accuracy and Cost.

  • System Accuracy

  • Machine Accuracy increasesas K increases

  • Human Accuracy decreasesas K increases

  • Cost increases as K increases


A quick glance at accuracy cost relation1

A quick glance at Accuracy-Cost Relation


There is an optimal cost that gives the highest accuracy

There is an optimal cost that gives the highest accuracy.


Towards a more realistic analysis of accuracy cost relationship

Towards a more realistic analysis of accuracy-cost relationship

  • With the above “assembly line” model, human accuracy and working time are influenced by a set of factors

    • K

    • Task difficulty

    • Expertise

      • I am familiar with office supplies, but not familiar with nuts and bolts.

    • Cognitive characteristics

      • Careful, smart, quick

      • Independent of the task


Use a probabilistic graphical model to capture the cognitive process of human classification

Use a probabilistic graphical model to capture the cognitive process of human classification

  • A probabilistic graphical model shows how the above different factors interact with each other, and influence the accuracy and cost.

  • Specifically, we use Bayesian Network, which characterizes the causal relationships of different factors.


Use a bayesian network to predict accuracy and cost

Use a Bayesian Network to predict accuracy and cost


Not only visually intuitive but also formal

Not only visually intuitive, but also formal


Inference and learning

Inference and learning

  • with this Bayesian Network, we predict the accuracy and cost by

  • Training data

    • A set of examples with known class labels

    • We let each human worker work on multiple examples, record the correctness and the working time

  • EM algorithm learns the parameters and the hidden variables


Usage of the model

usage of the model

  • Predict the accuracy-cost tradeoff

    • Given certain budget, what’s the highest accuracy we can achieve?

    • To achieve certain accuracy, what’s the lowest expected cost?

      • How to charge customers?

  • Optimally assign the workers to the tasks


Related works

Related Works

  • time and motion study

    • Scientific management (Taylorism)

  • Crowdsourcing

    • Amazon Mechanical Turk

    • learning worker expertise and accuracy

  • Item Response Theory

    • Psychometrics

      • IQ test, GRE, GMAT


Conclusion

Conclusion

  • In information age, we need a new “assembly line” to integrate human and machine intelligence.

  • We try to model human accuracy and working time by considering the interactions of a set of relevant factors, using a probabilisticgraphical model.

  • We use the model to predict the accuracy-costtradeoff, decide how to charge customers, and optimally assign tasks to human workers.


Thank you

Thank you!


  • Login