Embodied Conversational Agents:
This presentation is the property of its rightful owner.
Sponsored Links
1 / 34

Embodied Conversational Agents: A Case Study of Freudbot Bob Heller, PhD Athabasca University PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on
  • Presentation posted in: General

Embodied Conversational Agents: A Case Study of Freudbot Bob Heller, PhD Athabasca University November 3, 2004. Acknowledgements. Mike Proctor – AIML programmer Dean Mah – Web implementation Billy Cheung – Graphics, test chatter

Download Presentation

Embodied Conversational Agents: A Case Study of Freudbot Bob Heller, PhD Athabasca University

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Embodied Conversational Agents:

A Case Study of Freudbot

Bob Heller, PhD

Athabasca University

November 3, 2004


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Acknowledgements

Mike Proctor – AIML programmer

Dean Mah – Web implementation

Billy Cheung – Graphics, test chatter

Lisa Jewell – Chat log analysis, content developer, test chatter

Julianna Charchun – Chat log analysis

Jude Onuh – AIML programmer


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Embodied Conversational Agents

Definitions

  • Embodiment in Conversational Interfaces: REA (Cassel et al., 1999)

  • Embodied Conversational Agents (Cassel, Sullivan, Prevost, & Churchill, 2000)

    • FMTB model

      Vos (2002) offers 5 features of ECA

    • Human like appearance

    • Body used for communication purposes

    • Natural communication protocols

    • Multimodality

    • Social role


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Embodied Conversational Agents

Anthropomorphic Agents

Animated Interface Agents

Animated Pedagogical agents

Pedagogical Agent Persona

Intelligent Tutoring Systems

  • AutoTutor (Graesser et al) http://www.autotutor.org/index.htm

    Chatterbots or Chatbots

    - Weizenbaum’s (1966) Eliza


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Embodied Conversational Agents

Why?

  • primacy of conversation

  • Constructivist theory

  • The Media Equation

  • Persona effect

  • cognitive load


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Embodied Conversational Agents

Richard Wallace and A.L.I.C.E.

  • Artificial Linguistic Internet Computer Entity

    http://alicebot.org/

  • 3 time winner of the Loebner Contest (the holy grail for chatbots)

    http://www.loebner.net/

  • AIML – Artificial Intelligence Markup Language

    http://www.aimlbots.com/

  • PandoraBots

    http://www.pandorabots.com


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Embodied Conversational Agents

‘Theory’ behind ALICE

  • pattern matching

  • Zipf distribution

  • Iterative


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 1

Why Freud?

  • Initial plan of deployment

  • The famous personality application

    • Emile http://www.hud.ac.uk/hhs/research/emile/emileframeset.htm

    • Shakespeare http://www.pandorabots.com/pandora/talk?botid=c6937cfb3e354738

    • Hans Christian Anderson http://www.niceproject.com/about/

    • John Lennon


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 1

Developing the AIML

  • Narrative structure

  • Test chatters

  • How much ALICE?


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 1

Research Questions

  • Is it worth it?

  • Is ‘chattiness’ related to the subjective evaluation of chat experience?

  • Are there individual difference variables that are related to measures of chat performance/experience?


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 1: Methodology

  • Online Recruitment

    • restricted to psychology students

    • Incentive (1/30 chance at $300)

  • Random assignment to bot type

  • Controlled Chat

    • automatically directed to questionnaire after 10 mins of chat


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 1: Participants (N=67)

nPercent

GenderMen1218%

Women5582%

Age Distribution18-2269%

23-271522%

28-321116%

33-37710%

38-421522%

42+1319%

Student StatusFull-time2740%

Part-time3552%

Non-student58%

Self-rated academic Below avg00%

abilityAverage1319%

Above avg3958%

Excellent1522%


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Is it worth it?

  • self-report data*

Would you chat again?

Yes No

(n=30) (n=35)

2.7 1.8

3.4 1.6

3.2 1.8

3.4 1.9

3.4 2.1

3.6 2.2

4.1 2.8

Mean

Useful 2.2

Recommend2.4

Overall2.4

Enjoyable 2.6

Engaging2.7

Memorable2.8

Expansion3.4

* 5 point scale


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Is it worth it?

Best Features

Interactivity16

Able to ask questions with answers16

Learning about Freud & theories13

Simplicity/ease of use5

Entertaining/humorous5

Thought provoking5

No good features5

Technological features of Freudbot4

Potential to Freudbot4

Alternative learning style3

Novelty/uniqueness of Freudbot3

Tricking Freudbot2

Unpredictable2

Worst Features

Repetition33

Unable to answer questions23

Conversation did not flow12

Limited knowledge base10

User needed prior knowledge 3

User was uncertain about what to do 3

Not an effective learning tool3

Conversation was too short1

No sound1


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Is it worth it?

  • Chat logs

MeanRange

Number of Exchanges31.05-82

Mean

Proportion of on-task responses by participant* .60

questions.37

comments*.23

* correlated with a composite measure of self rated chat experience

Proportion of repetitions by Freudbot.25

Proportion of non-sensical by Freudbot .39


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Chattiness?

FreudAlice JustFreud

n=35n=32

Useful2.22.3

Recommend2.52.4

Overall2.52.4

Enjoyable2.7 2.6

Memorable3.02.7

Engaging2.82.7

Expansion3.33.5

# of Exchanges 32.2 29.7

On task Response* .56 .64

* -significant difference btw groups


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Individual difference variables?

  • demographic

    • Gender

    • Age

    • Student status*

    • Self-rated academic ability

  • computer experience & self-rated skill

  • academic background

    • # of university courses

    • # of distance ed courses*

    • # of psychology courses

    • Rated importance of Freud*


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Individual difference variables?

  • attitudes towards technology and education

    • Positive aspects of on-line activities

    • Independent Learner

    • negative aspects of on-line activities*


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 1 Summary

  • Is it worth it?

    • worth another look

  • Is ‘chattiness’ related to the subjective evaluation of chat experience?

    • ‘Chattiness’ is not the right level

    • Nass and Reeves (1998)

  • Are there individual difference variables that are related to measures of chat performance/experience?

    • some relations that make sense and others that don’t


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 2

Research Goals

1. Improve Performance

  • Fix repetition problem

  • Topic tags

  • More content

    2. Replication

    3. Instructional Set

    4. Future Development


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 2:Methodology

http://psych.athabascau.ca/html/Freudbot/test.html

  • online recruitment, incentive, & controlled chat identical to Freudbot 1

  • random assignment to instructional set

  • similar questionnaire with additional questions on applications and improvements


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Participants (N=55)

nPercent

GenderMen1018%

Women4582%

Age Distribution18-22713%

23-271731%

28-32713%

33-371120%

38-42611%

42+713%

Student StatusFull-time2647%

Part-time2851%

Non-student12%

Self-rated academic 0-5004%

ability50-6524%

66-791120%

80-893055%

90+1018%


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Improvement?

Would you chat again?

Yes No

(n=37) (n=18)

3.3 2.4

3.4 1.7

3.4 2.2

3.3 2.3

3.5 2.2

3.6 2.1

4.4 3.3

  • self-report data (5 point scale)

Freudbot 1 Freudbot 2

Useful**2.23.0

Recommend**2.42.9

Overall**2.43.0

Enjoyable 2.63.0

Engaging**2.73.1

Memorable2.83.1

Expansion**3.44.1

** - statisically significant


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Improvement?

  • Chat logs

MeanRange

Number of Exchanges28.43-115

Mean

Proportion of on-task responses by participant* .90

questions.36

comments.48

* correlated with a composite measure of self rated chat experience

Proportion of appropriate responses by Freud .60


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Replication?

  • Demographic

    • Gender*

    • Age

    • Student status*

    • Self-rated academic ability

  • computer experience

  • academic background

    • # of university courses

    • # of distance ed courses

    • # of psychology courses

    • Rated importance of Freud*


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Replication?

  • attitudes towards technology and education

    • Positive aspects of on-line activities

    • Independent Learner

    • negative aspects of on-line activities*


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Instructional Set?

Brief Set Elaborate Set n=27n=28

Useful3.12.9

Recommend2.82.9

Overall2.93.1

Enjoyable2.9 3.0

Memorable3.23.0

Engaging3.03.3

Expansion3.94.2

# of Exchanges 25.3 31.3

On task Response .90 .90


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Future Development?

Freudbot Improvements

Mean*

Chat behaviour4.2

Audio Response3.1

Voice Recognition2.6

Synchronization2.5

Animation/movment2.3

* 5-point scale

Other Applications

Mean*

Practice quizbot4.1

Famous personality4.1

Course content3.4

Chatroom3.3

Course Admin3.2


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Freudbot 2: Summary

1. Improvement

- yes, but clearly room for more

2. Replication

- some

3. Instructional Set

- no effects

4. Development


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Future Direction

  • Haptek Freud

    • Animacy/agency hypothesis

      http://psych.athabascau.ca/html/Freudbot/haptek.html

  • Piagetbot (Support from MCR)

    • learning outcomes

  • Skinnerbot (Lyle Grant)

  • Coursebot

  • Quizbot


Embodied conversational agents a case study of freudbot bob heller phd athabasca university

Questions?


  • Login