towards improved user product testing with cognitively enhanced scenarios
Download
Skip this Video
Download Presentation
Towards improved user-product testing with cognitively enhanced scenarios

Loading in 2 Seconds...

play fullscreen
1 / 31

Towards improved user-product testing with cognitively enhanced scenarios - PowerPoint PPT Presentation


  • 64 Views
  • Uploaded on

Towards improved user-product testing with cognitively enhanced scenarios. Wilfred van der Vegte. Where I work. Delft University of Technology (2011 figures) Founded in 1842 17,250 students 12 undergraduate programmes 33 post-graduate programmes 2,540 academic staff

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Towards improved user-product testing with cognitively enhanced scenarios' - sevita


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
towards improved user product testing with cognitively enhanced scenarios

Towards improved user-product testing with cognitively enhanced scenarios

Wilfred van der Vegte

where i work
Where I work

Delft University of Technology

(2011 figures)

  • Founded in 1842
  • 17,250 students
  • 12 undergraduate programmes
  • 33 post-graduate programmes
  • 2,540 academic staff
  • 8 faculties (departments)

Faculty ofIndustrial Design Engineering

Founded in 1969

Largest university-based design* school worldwide(*product design, consumer durables)

1,980 students

>4,700 alumni (MSc level)

3 departments,94 scientific staff

where i work1
Where I work

Section of Computer-Aided Design Engineering

  • ICT in product design
  • Historically: CAx
  • Currently: Cyber-physical systems, ubiquitous computing, smart products:how to apply technology in products & in product design
  • Headed by Prof. dr. mult. Imre Horváth
  • ~20 people, ~10 permanent scientificstaff
towards improved user product testing with cognitively enhanced scenarios1

Towards improved user-product testing with cognitively enhanced scenarios

Wilfred van der Vegte

involving humans in product simulations
Involving humans in product simulations

Real humans

(interactive simulation)

Virtual humans

(computer models)

human model

perception

Does not require human subjects

decision-making

(brain)

?

metabolism

motor control(brain, CNS)

action (muscles)

human model

computer simulation

computer simulation

product model

product model

simulating use with virtual humans
Simulating use with virtual humans

virtual human

perception

do we have to simulate

all of this?

decision-making

(brain)

scenario bundle

(by designer)

metabolism

motor control(brain, CNS)

approximate

action (muscles)

No ...let’s simplify

approximate

already solvedby others

the scenario bundle
Designer’s conjecture of human actions /decision-making

Graphical notation

Possible paths through use processconnecting interactions

Contains multiple options(courses of use process)

Controls a physics simulation of the product

Is specified as an automaton(←in this case a statechart)

The scenario bundle
example snack dispenser
Example: snack dispenser

scenario bundle

programming of product

simulation

advantages
Advantages
  • Use process of the product can be tested if only a computer model is available
  • No human subjects needed
  • Designer can do what-if studies
    • One scenario, multiple varieties of the product design
    • Varieties of how the product is programmed
    • Varieties in human behaviour, e.g., hesitation
    • Varieties of scenario: rearrange blocks & arrows
  • Building blocks of scenarios are reusable in other design projects
slide11
Key limitations
  • Simulation of physics computationally demanding and not (yet) very stable
  • Algorithms for realistic simulation of low-level human motor control exist, but have not been included
  • Lack of realism in human information processing (thinking, reasoning, ...): only what the designer preconceives is included

But ... Are animations of physics and human motor behavior always needed?

No! Often only the course of values of specific variables over time is important!(power consumption, temperature, speed, ...)

slide12
Demonstrative example: espresso machine with power-save function for water heating
  • Investigation of
  • boiler temperature
  • energy consumption
  • over a typical use process in which several coffee-making sessions take place, over several hours / days / weeks /...
  • with given settings for power saving (reduce power by x% after non-use over Δt = t1and by 100% after another Δt = t2)
slide18
To compensate for the missing 3D physics simulation, we have now used a hybrid automaton (discrete+continuous)

Models_and_specifications

Boiler

User

User_cupboard

Compute_input_time

Pump

2

Espresso_machine

Supplier

In addition, the statechart has become a timed hybrid automaton (THA) to efficiently deal with timing (latency, delays, time-outs,...)

Detect_power_change

no_heating_or_cooling

Supplier_processing

Espresso_machine_logic

OFF/discharge=0,P_pump=0,empty_sound=0

/T_water=T_amb

store_previous_value

[P_change>0]/t1=t

Coffee_serving

Boiler_thermostat

Track_consumption_remotely

T_last=T_amb

/P_prev=P_heat

[pump==0]

[pump==1]

heating_or_cooling

P_change=0

Prepare_orders

during:input_time=t-t1

[hasChanged(P_heat)==1]

Espresso_machine_physics

ON/P_pump=59

exit:input_time=0

Shipping

Pump

Reservoir

compute_P_change

1

detect_capsule

/P_change=P_heat-P_prev

[capsule_inserted==1]

Courier_service

Boiler

Cup_filling

present

absent

[P_change<0]/t1=t

T_last=T_water

[capsule_inserted==0]

/flow=4.29

/flow=5.86

[P_change>0]/t1=t

User

Compute_T_water

[capsule_inserted==0]

Run

2

build_up_pressure/

deliver

/discharge=flow

Exponential

after(2.5,sec)

during: T_water=T_amb+P_heat*k_1...

run_dry/discharge=0,

-(T_amb+P_heat*k_1-T_last)*ml.exp(-(k_3/m_water)*input_time)

[reservoir_content<5]

empty_sound=1

T_water = T_amb + P_heat*k1 - k2*exp[-(k3/m_water)*t];

k_2 = 1-(T_amb_P_heat*k_1-T_last)

[P_heat==0&&...

Constant_and_cold

T_water-T_amb<0.5]

[P_heat>0]

/T_water=T_amb

slide21
Energy consumption

(

kWh

)

Reservoir content

(

ml

)

Serving

User takes break

These simulation outcomes can be generated up to 5000 faster than real time

espressos

Serving lungos

User takes break

User refills reservoir

Boiler water temperature

(

K

)

Thermostat

controls

Power

-

save mode

temperature

Thermostat controls temperature

Power

-

save mode

User switches

machine off

Time

(

s

)

slide22
Revisiting the limitations
  • Simulation of physics has been simplified (no more 3D), and is fast and reliable
  • Low-level human motor control is disregarded, still the whole use process can be simulated (in this case)
  • Still lack of realism in human information processing (thinking, reasoning, ...): only what the designer preconceives is included
slide23
So ... how to increase realism in human information processing (thinking, reasoning, ...)?
  • Aspects of human information processing to be simulated
  • Logic of decision making:under which condition what action is taken?e.g. “IF cup is full THEN retrieve cup from machine”:straightforward execution of ‘normal’ use, assuming a particular history of preceding events.
  • But can a simulation predict a user acting according tothe production rule“IF cup is full THEN stick finger in it”? →unlikely!
slide24
So ... how to increase realism in human information processing (thinking, reasoning, ...)?
  • Aspects of human information processing to be simulated
  • Logic of decision making:under which condition what action is taken?e.g. “IF cup is full THEN retrieve cup from machine”:straightforward execution of ‘normal’ use (instructions), assuming a particular history of events, including required preceding actions.
  • We can howevertry to generatetypical aberrationsfrom‘regular use’:the so-callederror phenotypes
  • taxonomy accordingto Hollnagel→

error mode

simple phenotype

complex phenotype

(

applies to one action

)

(

applies to multiple

interconnected actions

)

repetition

restart

action in wrong place

reversal

jumping

omission

undershoot

action at wrong time

delay

premature action

action of wrong type

replacement

insertion

sidetracking

action not included in

capture

current plans

intrusion

branching

overshoot

slide25
So ... how to increase realism in human information processing (thinking, reasoning, ...)?
  • Aspects of human information processing to be simulated
  • Processing time:How long does it take to accomplisha given action, taking into accountaspectssuch as memory retrieval,memory capacity, learning,multitasking, distraction, etc.
  • These aspects can be simulatedusing cognitive architecturessuch as ACT-R
  • A cognitive architecture is
    • a blueprint of the human mind
    • based on findings from brain science
    • filled with psychologically validatedtask models expressed as production rules

intentional module

(

not identified

)

declarative module

goal buffer

(

temporal cortex

/

(

dorsolateral

hippocampus

)

prefrontal cortex

)

retrieval buffer

central production

(

ventrolateral

system

prefrontal cortex

)

(

basal ganglia

)

visual

motor

buffer

buffer

(

parietal

(

motor

cortex

)

cortex

)

motor

visual

module

module

(

motor

(

occipital

cortex

/

cortex

)

cerebel

-

lum

)

external world

how to realize simulations with cognitively enhanced scenarios ces
How to realize simulations with cognitively enhanced scenarios (CES)

intentional module

e

l

u

motor

motor

d

d

l

declarative module

goal buffer

r

o

o

buffer

module

central

m

w

l

goal

produc

-

l

a

a

n

n

buffer

tion

r

o

central production

i

e

t

t

retrieval buffer

system

visual

visual

n

x

system

e

e

t

buffer

module

n

i

motor

visual

buffer

buffer

declar

-

re

-

ative

trieval

visual

motor

module

buffer

module

module

external world

how to realize simulations with cognitively enhanced scenarios ces1
How to realize simulations with cognitively enhanced scenarios (CES)

error mode

simple phenotype

complex phenotype

applies to multiple

(

applies to one action

)

(

interconnected actions

)

repetition

restart

Human-ErrorPhenotype Generator

action in wrong place

reversal

jumping

omission

undershoot

action at wrong time

delay

premature action

action of wrong type

replacement

insertion

sidetracking

action not included in

capture

current plans

intrusion

branching

e

l

u

motor

motor

d

overshoot

d

l

r

o

o

buffer

module

central

m

w

l

goal

produc

-

l

a

a

n

n

buffer

tion

r

o

i

e

t

t

system

visual

visual

n

x

e

e

t

buffer

module

n

i

declar

-

re

-

ative

trieval

module

buffer

Scenario bundle (THA)

(human actions)

THA of the ‘world’(product, environment)

Cognitive Architecture (ACT-R)

slide28
Operation of ship locks –our first real-life application?

usually combined with a movable bridge

various types of boats and skippers

operator works remotely, usingmultiple monitors

gates, traffic lights, leveling, etc., are all

man-controlled

usually 2-3 chambers

slide29
Operation of ship locks –our first real-life application?

The Dutch Government Agency of Public Works & Water Management is developing a new system of centralized control rooms for locks, movable bridges and other objects

  • Potential problems:
  • Multitasking, but also boredom, can cause cognitive overload, cognitive lock-up& other error-invoking mental phenomena
  • Errors can cause injury, collisions or even flooding
  • With CES simulations the system can be pre-tested
  • without human subjects
  • batchwise, systematically varying parameters (weather, traffic density, ...)
  • to reveal incidents theoretically happening once in 10/100/1,000 years
  • that cannot be discovered through interactive testing
envisioned setup for first application
Envisioned setup for first application

error mode

simple phenotype

complex phenotype

applies to multiple

(

applies to one action

)

(

interconnected actions

)

repetition

restart

Human-ErrorPhenotype Generator

action in wrong place

reversal

jumping

omission

undershoot

action at wrong time

delay

premature action

action of wrong type

replacement

insertion

sidetracking

action not included in

capture

current plans

intrusion

branching

motor

motor

overshoot

buffer

module

central

goal

produc

-

buffer

tion

system

visual

visual

buffer

module

declar

-

re

-

ative

trieval

module

buffer

Scenario bundle (THA)

(human actions)

THA of the ‘world’(product, environment)

Cognitive Architecture (ACT-R)

conclusions
Conclusions

Simulation with cognitively enhanced scenarios (CES)

  • can help evaluating human-system interactions where many variations can influence the outcomes
  • based on psychologically validated knowledge on the workings of the human brain
  • benefits from simplifications & shortcuts:simulation speed is determined by slowest element in simulation loop →avoid complex models & algorithms
ad