When Users Pledge to Take Green Actions, Are They Solving a Decision Problem?

1. When Users Pledge to Take Green Actions, Are They Solving a Decision Problem? INFORMS Fall Conference, Washington, D.C. October 15, 2008

2. Acknowledgements • Funding: • Intel Corporation, “Leveraging Computational Technologies to Support Behavior Change”, Jennifer Mankoff, principal investigator • National Science Foundation grants NSF IIS-0745885 and NSF IIS-0205644, Jennifer Mankoff, principal investigator • Research assistance: • Victoria Yew StepGreen: Decisionmaking

3. Problem Motivation • Excessive energy consumption is a primary cause of global warming • Americans consumed 100 quadrillion BTUs of energy (U.S. Department of Energy 2006) • Energy consumption primarily linked to individual activities: lighting, heating and cooling (Bin and Dowlatabadi 2005, U.S. Environmental Protection Agency 2006) • Numerous Web-based initiatives exist to encourage environmentally responsive behavior • Daily actions: GreenSpeak.org • Green travel: EarthRoutes.net • Carbon footprint calculations and green living: Yahoo!Green • Social networking sites enable individuals to meet, collaborate and act collectively • Facebook • MySpace StepGreen: Decisionmaking

4. Research questions • Can we design an application to encourage reductions in daily activities that affect global warming that integrate: • On-line social networking, to attract people who are not necessarily ‘green’ • Displaying progress towards energy reduction goals, individually and in groups • Tracking actual energy consumption through sensors • Multiple technology platforms • Result is StepGreen, an initiative that combines scholarship, outreach and social change: • Mankoff, Matthews, Fussell and Johnson (2007) • Mankoff et al. (2008) StepGreen: Decisionmaking

5. StepGreen: A Web-based system in support individual action to combat global warming StepGreen.com actions for commitment Social network site ‘badge’ Cumulative effects of actions taken Source: www.stepgreen.org StepGreen: Decisionmaking

6. Alternative disciplinary views of StepGreen • Technology: a flexible, robust system will induce behavioral change and attract many users • IS/IT policy: a popular on-line application will provide novel insight into IT adoption, usage and outcomes. • Decision sciences: a Web-based decision support system will help ordinary users make better and more efficient decisions about high-impact daily actions • Public policy: learn about long-term social and environmental impacts of individual change as compared to national-level policy. StepGreen: Decisionmaking

7. Action selection is a decision problem • Behavioral change is a multi-step process: • Persuade users that action on a particular topic is urgent • Learn consequences of actions along multiple dimensions • Explore alternatives by examining tradeoffs in attribute space • Choose one or more actions that optimize utility • Observe actual impacts of actions • Update preferences for action attributes StepGreen: Decisionmaking

8. Action selection is a decision problem, cont’d • Action representation is important to decisionmaking: • Present choices in lists of varying sizes • One at a time • List • Present choices in differing ways: • Text descriptions • Tables, charts and graphs • Dynamics in action representations: • Static • Interactive What is the effect of length and information content of alternative visual representations of actions on decision time and decision quality? StepGreen: Decisionmaking

9. Theoretical Foundations • Human-computer interactions (Dix, et al. 2004) • Role of visual representation in decision-making • Lurie and Mason (2007) • Miller (2004) • Mandel and Johnson (2002) • Decision aids for large/complex decision problems • Payne et al. (1988) • Eiselt and Sandblom (2004) • Decision support systems for consumer choice • Häubl and Trifts (2000) • van der Heijden (2006) • Decision-making styles and barriers • Bruine de Bruin, Parker and Fischoff (2007) • Scott and Bruce (1995) StepGreen: Decisionmaking

10. Research gap: decision aids and DSS for public-sector problems • Limited literature on decision support and visualization especially by unsophisticated users, or those in vulnerable or underrepresented groups (Johnson 2006) Can specific decision aids and visualization strategies enable users to make decisions regarding lifestyle choices more effectively, or with higher levels of satisfaction? StepGreen: Decisionmaking

11. Previous experiment: design • Surveys: • Effects of human actions on global warming • Decision-making styles • Satisfaction with range of choices provided and rankings made • Evaluate text-based actions according to: • Length (‘terse’ vs. ‘verbose’) • Information content (‘relevant’ vs. ‘irrelevant’) • Action category (e.g. Heating, Lighting, Appliances, Water) • Actions are partitioned into two sets of non-dominated alternatives: • ‘Superior’ (8 or 10) • ‘Inferior’ (2 of 10) • Users rank top four actions out of 10 available in four categories StepGreen: Decisionmaking

12. Previous experiment: text representation Water Consumption StepGreen: Decisionmaking

13. Previous experiment: results • Decision quality shows significant variation with action categories • Few statistically significant associations with decision length or quality: • Question length • Information content • Decision style • Satisfaction with the range of decision alternatives • Satisfaction with the ranking decisions • No evidence of learning about environmental impacts StepGreen: Decisionmaking

14. Will alternative visualization methods make a difference? • Hypothesis: users prefer graphical representations of actions to text representations and will make better decisions. • New design: • Four action categories • Four visual representations of action characteristics and impacts: • ‘Terse’/’relevant’ text • Symbols • Value path • Bar chart StepGreen: Decisionmaking

15. Experiment data: classifications, values • Actions and categories: • Actions came from literature review and common sense • Categories inspired by ‘card-sorting’ exercises • Appliances • Heating/Cooling • Lighting and Appliances • Water Consumption • Impacts: • Carbon emissions • Change in energy usage: Energy Star (http://www.energystar.gov/) • Estimates of carbon savings: Energy Information Administration (http://www.eia.doe.gov/emeu/aer/txt/ptb1207b.html) • Dollar costs/savings: Energy Information Administration (http://www.eia.doe.gov/emeu/aer/txt/ptb0810.html) • Time costs/savings: rules of thumb • Quality of life: subjective assessments StepGreen: Decisionmaking

16. Actions: Text (Appliances) StepGreen: Decisionmaking

17. Actions: Symbols StepGreen: Decisionmaking

18. Actions: Value path StepGreen: Decisionmaking

19. Actions: Bar chart StepGreen: Decisionmaking

20. Research framework, cont’d • Users are assumed to act according to defined decision-making styles (Scott and Bruce 1995): • Intuitive • “When I make a decision I trust my inner feelings and reactions” • Rational • “I make decisions in a logical and systematic way” • Dependent • “I often need the assistance of other people when making important decisions” • Avoidant • “I often procrastinate when it comes to making important decisions” • Spontaneous • “When making decisions I do what seems natural at the moment” • Users learn about environmental impacts of various actions through choice process StepGreen: Decisionmaking

21. Hypotheses H 1: Participants’ decision quality and decision time vary according to action category. H2a: Graphical representation results in better outcomes than text representations H2b: Decision time and decision quality varies according to specific graphical representations H3a: ‘Rational’ decision-making styles are associated with higher-quality decisions H 3b: ‘Spontaneous’ decision-making styles are associated with lower-quality decisions H 3c: ‘Rational’ decision-making styles are associated with slower decisions H 3d: ‘’Intuitive’ decision-making styles are associated with more rapid decisions H4: Participants showed an increase in knowledge about impacts of specific actions with respect to global warming H 5: Gender is associated with decision quality and decision time StepGreen: Decisionmaking

22. Experiment design • Within-subjects design – four conditions (Martin 2004) • Survey software automatically randomized presentation orders and recorded decision times • Action categories, graphical representations counterbalanced across participants • 32 undergraduate and graduate student participants • Steps: • Study overview and consent forms • Pretest survey • Action choices • Posttest surveys • Compensation StepGreen: Decisionmaking

23. Results: Descriptive Statistics • Participant characteristics: • 72% male • 91% students (60% undergraduates; 31% graduate students) • 84.4% between 18 – 25 years old • Most live in households with unrelated mates and no children • Diverse racial, ethnic backgrounds • Decision-making styles (means of 1 – to – 5 scaled question responses within categories): • Rational: 3.82 • Intuitive: 3.59 • Dependent: 3.26 • Avoidant: 2.68 • Spontaneous: 2.87 StepGreen: Decisionmaking

24. Results: Descriptive Statistics, cont’d Baseline global warming knowledge is generally low StepGreen: Decisionmaking

25. Results: Outcome measures • Dominated responses = number of choices that were dominated in the subject’s responses = 0, 1, 2 • Response time = time to make all selections; used log-transformed times due to skewed original values StepGreen: Decisionmaking

26. Results: Effect of action category Mixed model analyses showed no difference between domains (appliances, heating, lighting, water) for: • Number of dominated choices (F [3, 36.3] = 1.71, p = .17) or • Log response time (F [3, 58.93] = 1.13, p = .35) No effect on decision quality or time due to action category StepGreen: Decisionmaking

27. Result: Effect of representation type • Collapsed outcomes over action categories, giving each participant one score for each visualization condition • Dominated choices: • Log response time: Representation type has no effect on decision quality Response times longer for text than for graphics, and response times do not differ by graphic type StepGreen: Decisionmaking

28. Results: Pre- and post-test learning After experiment, users generally perceived greater global warming impact on all actions StepGreen: Decisionmaking

29. Correlations: user characteristics and decision outcomes • Mean number of dominated choices per trial negatively correlated with log of response time ( -0.405 [0.021]) • Gender (male = 0, women = 1) is negatively correlated with log of response time (-0.439 [0.012]) • Decision-making style has no statistically significant effect on decision-making quality or decision time • Gender not statistically significantly correlated with decision style or decision quality StepGreen: Decisionmaking

30. Correlations: graphical representations and outcomes • Some relationships between decision times across graphical representations: • Text and value path (.470 [0.008]) • Text and bar chart (.513 [0.010]) • Symbols and value path (.603 [0.000]) • Symbols and bar chart (.403 [0.051]) StepGreen: Decisionmaking

31. So, are StepGreen users solving a decision problem? • We think so..but decision context so far provides little support. • For static representations of actions: • Generally, neither length of action descriptions, information content within descriptions or graphical representation of actions had significant effects on decision outcomes. • Why did more information about decision alternatives not help subjects make better decisions? • Alternatives ranking too demanding cognitively? • Information insufficiently tailored to different needs, interests and backgrounds of subjects? StepGreen: Decisionmaking

32. Implications for research • Design: • For maximum speed, emphasize graphical representations of actions • Particular graphical representation not important • Next steps: • Use a human intermediary to help users choose actions • Apply new methods to measure decision-making competence StepGreen: Decisionmaking

33. New Experiment: Human-assisted decisionmaking • Inspiration: • Risk-perception literature (Florig, et al. 2001): detailed problem representation improves risk ranking • Decision competence literature (Parker, Bruine de Bruin and Fischoff 2007), in which measures of decision efficacy are associated with satisfaction with decisions made • Idea: • Use a human intermediary to help users better understand their own values and preferences and characteristics of action alternatives StepGreen: Decisionmaking

34. Goal: evaluate decision outcomes along two dimensions of interventions • Intermediary types: • ‘Peer’ intermediary - informal, youthful affect and use a minimum of technical language • ‘Expert’ intermediary - more formal, academic affect, use technical language and appear to be an authority on behavioral changes and impacts of actions on climate change. • Intermediation type: • Quantitative - Problem-focused, scientific presentation of the impacts of various actions using figures and descriptions of relevant calculations • Qualitative - individual-focused, interactive, holistic discussion using probing questions to learn about subject attitudes regarding different actions StepGreen: Decisionmaking

35. Scientific versus informal presentation of actions • Scientific presentation: • Diagrams will convey the mechanisms by which actions will result in energy savings and a reduction in carbon emissions. • Equations will convey the means by which energy savings and reductions in carbon emissions are computed for ‘typical’ users. • No mention will be made of individual preferences for some classes of actions over others, or the means by which individual lifestyle characteristics influence the impacts of various actions. StepGreen: Decisionmaking

36. Scientific versus informal presentation of actions, continued • Informal presentation: • Scripted questions to subjects will determine • Categories of actions are most important to them • Constraints that limit consideration of certain action • Motivation for pursuing energy reducing actions • No mention will be made of amounts of energy saved for various actions, or the means by which impacts are computed. StepGreen: Decisionmaking

37. Proposed data analysis • Descriptive statistics: • Demographics • Measures of decision-making styles • Decision-making competency • Decision-making outcomes • Hypothesis tests: • Impact on decision-making outcomes of • Intermediary type - intermediation type pairs • Decision-making styles • Decision-making competency StepGreen: Decisionmaking

38. See you next year! Questions? StepGreen: Decisionmaking

39. References • Bin, S. and H. Dowlatabadi. 2005. Consumer Lifestyle Approach to US Energy Use and the Related CO2 Emissions. Energy Policy33: 197 – 208. • Bruine de Bruin, W., Parker, A.M. and B. Fischhoff. 2007. Individual Differences in Adult Decision-Making Competence. Journal of Personality and Social Psychology92(5): 938 – 956. • Häubl, G. and V. Trifts. 2000. Consumer Decision Making in Online Shopping Environments: The Effects of Interactive Decision Aids. Marketing Science19(1): 4 – 21. • Lurie, N.H. and C.H. Mason. 2007. Visual Representation: Implications for Decision Making. Journal of Marketing71: 160 – 177. • Mankoff, J., Fussell, S.R., Johnson, M.P., Matthews, D., Blais, D., Dillahunt, T., Glaves, R., McGuire, R., Setlock, L., Schick, A. Thompson, R. and H.-C. Wang. 2008. “StepGreen: Engaging Individuals in Energy-Saving Actions Online.” Under review for presentation at Computer/Human Interaction Conference 2009, Boston, MA. • Mankoff, J., Matthews, D., Fussell, S.R. and M. Johnson. 2007. “Leveraging Social Networks to Motivate Individuals to Reduce Their Ecological Footprints”, in Proceedings of the 40th Annual Hawaii International Conference on System Sciences (CD-ROM), January 3 – 6, 2007, Computer Society Press, 2007 (10 pages) • Scott, S.G. and R.A. Bruce. 1995. Decision Making Style: The Development and Assessment of a New Measure. Educational and Psychological Measurement55: 818 – 31. • U.S. Department of Energy. 2006. Annual Energy Review 2005. Washington, D.C.: Energy Information Administration, DOE/EIA-0384. World Wide Web: http://tonto.eia.doe.gov/FTPROOT/multifuel/038405.pdf. • van der Heijden, H. 2006. Mobile Decision Support for In-Store Purchase Decisions. Decision Support Systems42: 656 – 663. StepGreen: Decisionmaking

40. Previous experiment: choice sets Heating/Cooling StepGreen: Decisionmaking

41. Correlations: User characteristics and decision outcomes StepGreen: Decisionmaking

42. Correlations: Action representations and decision outcomes StepGreen: Decisionmaking