Towards improved user product testing with cognitively enhanced scenarios
This presentation is the property of its rightful owner.
Sponsored Links
1 / 31

Towards improved user-product testing with cognitively enhanced scenarios PowerPoint PPT Presentation


  • 42 Views
  • Uploaded on
  • Presentation posted in: General

Towards improved user-product testing with cognitively enhanced scenarios. Wilfred van der Vegte. Where I work. Delft University of Technology (2011 figures) Founded in 1842 17,250 students 12 undergraduate programmes 33 post-graduate programmes 2,540 academic staff

Download Presentation

Towards improved user-product testing with cognitively enhanced scenarios

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Towards improved user product testing with cognitively enhanced scenarios

Towards improved user-product testing with cognitively enhanced scenarios

Wilfred van der Vegte


Where i work

Where I work

Delft University of Technology

(2011 figures)

  • Founded in 1842

  • 17,250 students

  • 12 undergraduate programmes

  • 33 post-graduate programmes

  • 2,540 academic staff

  • 8 faculties (departments)

Faculty ofIndustrial Design Engineering

Founded in 1969

Largest university-based design* school worldwide(*product design, consumer durables)

1,980 students

>4,700 alumni (MSc level)

3 departments,94 scientific staff


Where i work1

Where I work

Section of Computer-Aided Design Engineering

  • ICT in product design

  • Historically: CAx

  • Currently: Cyber-physical systems, ubiquitous computing, smart products:how to apply technology in products & in product design

  • Headed by Prof. dr. mult. Imre Horváth

  • ~20 people, ~10 permanent scientificstaff


Towards improved user product testing with cognitively enhanced scenarios1

Towards improved user-product testing with cognitively enhanced scenarios

Wilfred van der Vegte


Typical engineering simulations

Typical engineering simulations


Involving humans in product simulations

Involving humans in product simulations

Real humans

(interactive simulation)

Virtual humans

(computer models)

human model

perception

Does not require human subjects

decision-making

(brain)

?

metabolism

motor control(brain, CNS)

action (muscles)

human model

computer simulation

computer simulation

product model

product model


Simulating use with virtual humans

Simulating use with virtual humans

virtual human

perception

do we have to simulate

all of this?

decision-making

(brain)

scenario bundle

(by designer)

metabolism

motor control(brain, CNS)

approximate

action (muscles)

No ...let’s simplify

approximate

already solvedby others


The scenario bundle

Designer’s conjecture of human actions /decision-making

Graphical notation

Possible paths through use processconnecting interactions

Contains multiple options(courses of use process)

Controls a physics simulation of the product

Is specified as an automaton(←in this case a statechart)

The scenario bundle


Example snack dispenser

Example: snack dispenser

scenario bundle

programming of product

simulation


Advantages

Advantages

  • Use process of the product can be tested if only a computer model is available

  • No human subjects needed

  • Designer can do what-if studies

    • One scenario, multiple varieties of the product design

    • Varieties of how the product is programmed

    • Varieties in human behaviour, e.g., hesitation

    • Varieties of scenario: rearrange blocks & arrows

  • Building blocks of scenarios are reusable in other design projects


Towards improved user product testing with cognitively enhanced scenarios

Key limitations

  • Simulation of physics computationally demanding and not (yet) very stable

  • Algorithms for realistic simulation of low-level human motor control exist, but have not been included

  • Lack of realism in human information processing (thinking, reasoning, ...): only what the designer preconceives is included

But ... Are animations of physics and human motor behavior always needed?

No! Often only the course of values of specific variables over time is important!(power consumption, temperature, speed, ...)


Towards improved user product testing with cognitively enhanced scenarios

Demonstrative example: espresso machine with power-save function for water heating

  • Investigation of

  • boiler temperature

  • energy consumption

  • over a typical use process in which several coffee-making sessions take place, over several hours / days / weeks /...

  • with given settings for power saving (reduce power by x% after non-use over Δt = t1and by 100% after another Δt = t2)


Towards improved user product testing with cognitively enhanced scenarios

Building a scenario bundle for using the espresso machine


Towards improved user product testing with cognitively enhanced scenarios

Building a scenario bundle for using the espresso machine


Towards improved user product testing with cognitively enhanced scenarios

Building a scenario bundle for using the espresso machine


Towards improved user product testing with cognitively enhanced scenarios

Building a scenario bundle for using the espresso machine


Towards improved user product testing with cognitively enhanced scenarios

Building a scenario bundle for using the espresso machine

User


Towards improved user product testing with cognitively enhanced scenarios

To compensate for the missing 3D physics simulation, we have now used a hybrid automaton (discrete+continuous)

Models_and_specifications

Boiler

User

User_cupboard

Compute_input_time

Pump

2

Espresso_machine

Supplier

In addition, the statechart has become a timed hybrid automaton (THA) to efficiently deal with timing (latency, delays, time-outs,...)

Detect_power_change

no_heating_or_cooling

Supplier_processing

Espresso_machine_logic

OFF/discharge=0,P_pump=0,empty_sound=0

/T_water=T_amb

store_previous_value

[P_change>0]/t1=t

Coffee_serving

Boiler_thermostat

Track_consumption_remotely

T_last=T_amb

/P_prev=P_heat

[pump==0]

[pump==1]

heating_or_cooling

P_change=0

Prepare_orders

during:input_time=t-t1

[hasChanged(P_heat)==1]

Espresso_machine_physics

ON/P_pump=59

exit:input_time=0

Shipping

Pump

Reservoir

compute_P_change

1

detect_capsule

/P_change=P_heat-P_prev

[capsule_inserted==1]

Courier_service

Boiler

Cup_filling

present

absent

[P_change<0]/t1=t

T_last=T_water

[capsule_inserted==0]

/flow=4.29

/flow=5.86

[P_change>0]/t1=t

User

Compute_T_water

[capsule_inserted==0]

Run

2

build_up_pressure/

deliver

/discharge=flow

Exponential

after(2.5,sec)

during: T_water=T_amb+P_heat*k_1...

run_dry/discharge=0,

-(T_amb+P_heat*k_1-T_last)*ml.exp(-(k_3/m_water)*input_time)

[reservoir_content<5]

empty_sound=1

T_water = T_amb + P_heat*k1 - k2*exp[-(k3/m_water)*t];

k_2 = 1-(T_amb_P_heat*k_1-T_last)

[P_heat==0&&...

Constant_and_cold

T_water-T_amb<0.5]

[P_heat>0]

/T_water=T_amb


Towards improved user product testing with cognitively enhanced scenarios

User


Towards improved user product testing with cognitively enhanced scenarios

Energy consumption

(

kWh

)

Reservoir content

(

ml

)

Serving

User takes break

These simulation outcomes can be generated up to 5000 faster than real time

espressos

Serving lungos

User takes break

User refills reservoir

Boiler water temperature

(

K

)

Thermostat

controls

Power

-

save mode

temperature

Thermostat controls temperature

Power

-

save mode

User switches

machine off

Time

(

s

)


Towards improved user product testing with cognitively enhanced scenarios

Revisiting the limitations

  • Simulation of physics has been simplified (no more 3D), and is fast and reliable

  • Low-level human motor control is disregarded, still the whole use process can be simulated (in this case)

  • Still lack of realism in human information processing (thinking, reasoning, ...): only what the designer preconceives is included


Towards improved user product testing with cognitively enhanced scenarios

So ... how to increase realism in human information processing (thinking, reasoning, ...)?

  • Aspects of human information processing to be simulated

  • Logic of decision making:under which condition what action is taken?e.g. “IF cup is full THEN retrieve cup from machine”:straightforward execution of ‘normal’ use, assuming a particular history of preceding events.

  • But can a simulation predict a user acting according tothe production rule“IF cup is full THEN stick finger in it”? →unlikely!


Towards improved user product testing with cognitively enhanced scenarios

So ... how to increase realism in human information processing (thinking, reasoning, ...)?

  • Aspects of human information processing to be simulated

  • Logic of decision making:under which condition what action is taken?e.g. “IF cup is full THEN retrieve cup from machine”:straightforward execution of ‘normal’ use (instructions), assuming a particular history of events, including required preceding actions.

  • We can howevertry to generatetypical aberrationsfrom‘regular use’:the so-callederror phenotypes

  • taxonomy accordingto Hollnagel→

error mode

simple phenotype

complex phenotype

(

applies to one action

)

(

applies to multiple

interconnected actions

)

repetition

restart

action in wrong place

reversal

jumping

omission

undershoot

action at wrong time

delay

premature action

action of wrong type

replacement

insertion

sidetracking

action not included in

capture

current plans

intrusion

branching

overshoot


Towards improved user product testing with cognitively enhanced scenarios

So ... how to increase realism in human information processing (thinking, reasoning, ...)?

  • Aspects of human information processing to be simulated

  • Processing time:How long does it take to accomplisha given action, taking into accountaspectssuch as memory retrieval,memory capacity, learning,multitasking, distraction, etc.

  • These aspects can be simulatedusing cognitive architecturessuch as ACT-R

  • A cognitive architecture is

    • a blueprint of the human mind

    • based on findings from brain science

    • filled with psychologically validatedtask models expressed as production rules

intentional module

(

not identified

)

declarative module

goal buffer

(

temporal cortex

/

(

dorsolateral

hippocampus

)

prefrontal cortex

)

retrieval buffer

central production

(

ventrolateral

system

prefrontal cortex

)

(

basal ganglia

)

visual

motor

buffer

buffer

(

parietal

(

motor

cortex

)

cortex

)

motor

visual

module

module

(

motor

(

occipital

cortex

/

cortex

)

cerebel

-

lum

)

external world


How to realize simulations with cognitively enhanced scenarios ces

How to realize simulations with cognitively enhanced scenarios (CES)

intentional module

e

l

u

motor

motor

d

d

l

declarative module

goal buffer

r

o

o

buffer

module

central

m

w

l

goal

produc

-

l

a

a

n

n

buffer

tion

r

o

central production

i

e

t

t

retrieval buffer

system

visual

visual

n

x

system

e

e

t

buffer

module

n

i

motor

visual

buffer

buffer

declar

-

re

-

ative

trieval

visual

motor

module

buffer

module

module

external world


How to realize simulations with cognitively enhanced scenarios ces1

How to realize simulations with cognitively enhanced scenarios (CES)

error mode

simple phenotype

complex phenotype

applies to multiple

(

applies to one action

)

(

interconnected actions

)

repetition

restart

Human-ErrorPhenotype Generator

action in wrong place

reversal

jumping

omission

undershoot

action at wrong time

delay

premature action

action of wrong type

replacement

insertion

sidetracking

action not included in

capture

current plans

intrusion

branching

e

l

u

motor

motor

d

overshoot

d

l

r

o

o

buffer

module

central

m

w

l

goal

produc

-

l

a

a

n

n

buffer

tion

r

o

i

e

t

t

system

visual

visual

n

x

e

e

t

buffer

module

n

i

declar

-

re

-

ative

trieval

module

buffer

Scenario bundle (THA)

(human actions)

THA of the ‘world’(product, environment)

Cognitive Architecture (ACT-R)


Towards improved user product testing with cognitively enhanced scenarios

Operation of ship locks –our first real-life application?

usually combined with a movable bridge

various types of boats and skippers

operator works remotely, usingmultiple monitors

gates, traffic lights, leveling, etc., are all

man-controlled

usually 2-3 chambers


Towards improved user product testing with cognitively enhanced scenarios

Operation of ship locks –our first real-life application?

The Dutch Government Agency of Public Works & Water Management is developing a new system of centralized control rooms for locks, movable bridges and other objects

  • Potential problems:

  • Multitasking, but also boredom, can cause cognitive overload, cognitive lock-up& other error-invoking mental phenomena

  • Errors can cause injury, collisions or even flooding

  • With CES simulations the system can be pre-tested

  • without human subjects

  • batchwise, systematically varying parameters (weather, traffic density, ...)

  • to reveal incidents theoretically happening once in 10/100/1,000 years

  • that cannot be discovered through interactive testing


Envisioned setup for first application

Envisioned setup for first application

error mode

simple phenotype

complex phenotype

applies to multiple

(

applies to one action

)

(

interconnected actions

)

repetition

restart

Human-ErrorPhenotype Generator

action in wrong place

reversal

jumping

omission

undershoot

action at wrong time

delay

premature action

action of wrong type

replacement

insertion

sidetracking

action not included in

capture

current plans

intrusion

branching

motor

motor

overshoot

buffer

module

central

goal

produc

-

buffer

tion

system

visual

visual

buffer

module

declar

-

re

-

ative

trieval

module

buffer

Scenario bundle (THA)

(human actions)

THA of the ‘world’(product, environment)

Cognitive Architecture (ACT-R)


Conclusions

Conclusions

Simulation with cognitively enhanced scenarios (CES)

  • can help evaluating human-system interactions where many variations can influence the outcomes

  • based on psychologically validated knowledge on the workings of the human brain

  • benefits from simplifications & shortcuts:simulation speed is determined by slowest element in simulation loop →avoid complex models & algorithms


  • Login