Broader Impacts

1. Research you can use Broader Impacts Judith Olson University of California Irvine

2. Today I’m going to cover…. • Three recent events that inspired this “call to arms” • Your role in this discussion • What it means to have impact • Kinds of impact • Recap of what it means to have impact • Scope, Cost, Timeline… • Your pledge about making an impact

3. Collaboration Success Wizard • Theory based on • The literature on teams • Own own observations and interviews of over 50 • Science Collaborations • Corporate virtual teams • To verify theory • Need data • Online survey with advice to motivate participation • They get the help and we get the data

4. Collaboration Success Wizard • Web accessible assessment tool • Assesses • Strengths • Challenges • How to overcome the challenges

5. We are having an impact • NSF • Had us give a talk to Federal funders in general • “I have needed this for the last 10 years! Thank you.” • Teams who were assessed welcomed advice • “It drew out patterns in the way our members work that we were not conscious of, confirmed some of our impressions, and allowed us to hear frankly from our members. • …useful as an independent evaluation tool not tied to a funding agency or other review panel”.

6. Clinical and Translational Science Awards (CTSA) • Object of study • …”to speed the translation of laboratory discoveries into treatment for patients.” • “from bench to bedside.” • National Institute of Health: • 60 CTSA awards in 30 states plus DC

7. Clinical and Translational Science Awards National Institutes of Health Since 2006 $733 M

8. NSF “Dear Colleague” “While most researchers know what is meant by Intellectual Merit, experience shows that many researchers have a lessthan clear understanding of the meaning of Broader Impacts.”

9. Impact • Many of us came to this field to change the digital world • Technology had gone awry • Many early people attracted to HCI were “Children of the 60s”

10. Then… • Our careers were caught up in the reward structures • Industry • Create new products • Disincentive to makefindings available to others • Academia • Publish new findings • Stay on topic, build a reputation

11. Worry….. • Where have all the impacts gone • Long time passing

12. How we will proceed…. • I will describe what I think it means to have impact • I will list a number of ways we do and can have an impact • You pledge… • The card on your seat • What other ways can you have impact • How are you going to have an impact • Collected by SVs at the door as you leave

13. What it means to have impact • What “counts” • Theory gets used • Downloads/views • Profits • Degrees/Education • Technologies • Lives changed • ….. • Who is impacted? • Students • Developers • Consultants • Specific populations • The general public

14. What it means to have impact • Scopes differ • You affect some people directly • Interventions, teaching • You enable others to be better at making better products • Toolkits • You set policy • Affect a large number of people

15. What it means to have impact • Time scales differ • Now • e.g. Action research • 1-3 years • e.g., Publications • 20-30 years • e.g., TheoryAssessment Tools • 40-50 years • e.g., Cyberinfrastructure development • ? • e.g., Policy (like SOPA/PIPA)

16. What it means to have impact • Access? • Free • Toolkits… • Wizard • …. • Fees • Commercial Assessment Tools • Products • Educational degree • ….

17. Kinds of Impacts • Theories • Assessment tools • Technological innovations • Guidelines, templates, patterns, toolkits and standards • Policies • New media dissemination • Action research • Teaching and teaching materials • … What else? Which kinds of impacts will YOU make?

18. Theories • “There is nothing so practical as a good theory” Kurt Lewin • “He who loves practice without theory • is like the sailor who boards ship without a rudder and compass and • never knows where he may cast” Leonardo Da Vinci

19. Theories • Who • Other researchers • Consultants • Tool developers • How • Read and build on/test theory • Scope • Small at first • Time scale • 1-3 or more years • Access • Free

20. Kinds of Impacts • Theories • Assessment tools • Technological innovations • Guidelines, templates, patterns, toolkits and standards • Policies • New media dissemination • Action research • Teaching and teaching materials What else? Which kinds of impacts will YOU make?

21. Theories delivered as Assessment Tools • Collaboration Success Wizard • Globesmart • Myers-Briggs Personality Assessment • CogTool • …

22. Assessment Tools: GlobeSmart • Based on academic theories of cultural differences • David Matsumoto • Handbook of Culture and Psychology • Like the Wizard, they collect data to adjust their assessments • Recent upgrade used data from 400,000 users from over 60 countries

23. Assessment Tools: Globesmart

24. Assessment Tools: GlobeSmart

25. Assessment Tools: GlobeSmart

26. Assessment Tools: GlobeSmart

27. Assessment Tools: Myers Briggs • Based on the work of Carl Jung • Developed further by Myers and Briggs • Like the GlobeSmart you can see differences in values and habits with people you interact with • Dimensions of discussion • Some professional help

28. Assessment Tools: CogTool • Based on work of Bonnie John • Based on Card, Moran, & Newell • GOMS and the Model Human Processor • A general purpose UI prototyping tool • It automatically evaluates your design with a predictive human performance model • A “cognitive crash dummy”

29. Assessment Tools: CogTool

30. Assessment Tools: CogTool • “You can compare expert use task time without recruiting participants…An excellent choice for completely new systems that don’t already have experts.”

31. Assessment Tools • Who • General public • How • Take the assessment • Scope • Could be huge • Time scale • Immediate • Access • Some are free; some cost money

32. Kinds of Impacts • Theories • Assessment tools • Technological innovations • Guidelines, templates, patterns, toolkits and standards • Policies • New media dissemination • Action research • Teaching and teaching materials What else? Which kinds of impacts will YOU make?

33. Technical Innovations: Alice • 3-D programming environment • For telling a story • Playing an interactive game • Teaching tool for introductory programming • Formally shown to improve learning and performance Randy Pausch

34. Technical Innovations: Alice Caitlin Kelleher, 2006

35. Technical Innovations: Alice • Using storytelling to make computer programming attractive to middle school girls • Storytelling Alice users • spent 42% more time programming • were more than three times as likely to sneak in extra time to continue working on their programs Caitlin Kelleher

36. Technical Innovations: Alice • 10% of the nation’s colleges now use Alice • An accompanying textbook, lessons, test banks • 88% of “at risk” students who had Alice in a pre-CS1 course were retained through CS2 • 3.03 GPA

37. Which then inspired…. • iMuse • A requirements engineering environment where both developers and stakeholders could understand the flow Kristina Winbladh

38. Technical Innovations: More... • Hypertext Transfer Protocol HTTP • HTTP/1.1 spec • Fielding, Gettys, Mogul, Frystyk and Berners-Lee • WebDAV extension • “Architecture of the Web” • Fielding and Taylor

39. Technical Innovations: More…. • Aspect Oriented Programming • Difference lies in the power, safetyand usability of the constructs provided • Original article downloaded6,681 times • 16,600 articles in Google Scholar with “Aspect Oriented Programming” Crista Lopes

40. Technical innovations • Who • Students • The general public • Other developers • How • Use the technology that makes things possible • Scope • Huge • Time scale • 5-10 years • Access • Often free (though products cost money)

41. Kinds of Impacts • Theories • Assessment tools • Technological innovations • Guidelines, templates, patterns, toolkits and standards • Policies • New media dissemination • Action research • Teaching and teaching materials What else? Which kinds of impacts will YOU make?

42. Guidelines, templates, and patterns • All provide conventions • So there is little new to learn • Where things go, what they look like • Sometimes task flow guide

43. Guidelines, templates, and patterns • What are they based on? Are they consistent? (Human Interface Guidelines)

44. Guidelines, templates, and patterns • Principles, patterns and practices for improvinguse experience • Early instance: Christopher Alexander Christian Crumlish & Erin Malone

45. Guidelines, templates, and patterns • Their effectiveness depends on • The research they are based on • The context in which they arose • Their fit to the context they are being applied to

46. Toolkits • UI Development environments • With extra features • Highly interactive • Graphical • Direct manipulation • Automatic undo • Support for animation • Gesture recognition • Amulet - C++ • Garnet – Common Lisp, X11, and Mac Brad Myers

47. Standards Keeping these up to date….

48. Guidelines, templates, patterns, toolkits and standards • Who • Developers • End users • How • Find and use relevant templates…. • Scope • Speeds development, makes software consistent • Time scale • Immediate • Access • Free

49. Kinds of Impacts • Theories • Assessment tools • Technological innovations • Guidelines, templates, patterns, toolkits and standards • Policies • New media dissemination • Action research • Teaching and teaching materials What else? Which kinds of impacts will YOU make?

50. Policies • Stop Online Piracy Act (SOPA)/Protect Intellectual Property Act (PIPA) • Network neutrality • Participatory design in Scandinavia • Open access vs. commercial production of educational materials • Data sharing policies • ……