Data Analysis, Interpretation, and Presentation

1. Data Analysis, Interpretation, and Presentation Devin Spivey Asmae Mesbahi El Aouame Rahul Potghan

2. Objectives • Difference between qualitative and quantitative data and analysis. • Analyze data gathered from questionnaires. • Analyze data gathered from interviews. • Analyze data gathered from observation studies. • Software packages for data analysis. • Pitfalls in data analysis, interpretation, and presentation. • Presenting your findings.

3. Qualitative and quantitative • Quantitative:numbers, translated into numbers. • Qualitative: difficult to express in numerical terms in a sensible fashion. Be careful in translating qualitative data into quantitative data.

4. Types of Data

5. Simple quantitative analysis • Simple quantitative analysis techniques: • Percentages:for standardizing data (compare large sets of data). • Averages: • Mean: commonly understood average. • Median: middle value of the data. • Mode: most commonly occurring number. • Initial Analysis: finding averages, outliers, depict any patterns from the graphical representation of data on a spreadsheet.

6. Initial analysis example: data gathering • Evaluation study of an e-commerce website: identify transactions’ difficulties faced by users. • Data gathering methods: • Questionnaires. • Observation of a controlled task. • data logging.

7. Initial analysis example: Visualization of data

8. Initial analysis example: Visualization of data • Outlier: • Removed from the larger data set since it distorts the general patterns. • Interesting case for further analysis.

9. Activity(1) • The data in the table represents the time taken by a group of users to select and buy an item from an online shopping website.

10. Activity(2) • Further investigation: • The values for users N(24) and S(26) are higher than the others. • Trends: • - Users at the beginning of the testing time performed faster than those towards the end of the testing. • - O is at the end of the testing but performed well.

11. How question design affects data analysis

12. Comparingtwoproducts: 2 Phone designs

13. Initial analysis from data logs (1) • Research question: investigate how effective Massively Multiplayer Online Role-Playing Games (MMORPGs) are at encouraging interactivity between users. • Data: data logs and video recordings of players interactions in SWG. • Ethnography study to identify the locations heavily used by players.

14. Initial analysis from data logs (2)

15. Simple Qualitative Analysis • Qualitative analysis –expresses the nature of elements and is represented as themes, patterns, stories • Qualitative data: difficult to measure sensibly as numbers.e.g. counting number of words to measure dissatisfaction .

16. -The first step is to gain an overall impression of the data and start looking for patterns.- Next comes more detailed work using structured frameworks or theories to support the investigation.- Patterns may relate to a variety of aspects. e.g. behavior, user groups, places or situations where certain events happen.

17. In terms of structure:-Unstructured: Not directed by a script. Rich but not replicable. e.g. video material. -Structured: Tightly scripted. Replicable but may lack richness.e.g. questionnaire. -Semi-structured: Guided by a script but interesting issues can be explored in more depth. Provides a good balance between richness and replicability.e.g. interviews.

18. When Should I Use Qualitative Vs. Quantitative Research? http://www.youtube.com/watch?v=638W_s5tRq8&feature=related

19. Three types of qualitative analysis 1. Identifying recurring patterns and themes2. Categorizing data3. Analyzing critical incidents

20. 1. Identifying recurring patterns and themes

21. 1. Identifying recurring patterns and themes • Emergent from data, dependent on observation framework if used • Patterns in Quantitative can be find identified by graphical representation but for Qualitative, requires researcher to be immersed in data • Studying the data • Focusing on the study goals • Keeping clear records of the analysis as it progresses and close description of themes or patterns that are emerging. • e.g. Box 8.4, Themes in European culture.

22. 2. Categorizing data • Can be at a high level of detail such as identifying stories or themes OR at a fine level of detail in which each word, phrase or gesture is analyzed. • Most challenging aspects :1. Determining meaningful categories that are orthogonal (do not overlap each other in any way).2. Deciding on the appropriate granularity for the categories (word, phrase, sentence, or paragraph level). • The categorization scheme used must be reliable so that the analysis can be replicated! -Categorization scheme may be emergent or pre-specified

23. 3. Looking for critical incidents • It is a flexible set of principles that emerged from work carried out in the United States Army Air Forces where the goal was to identify the critical requirements of “good” and “bad” performance by pilots. • Two basic principles:1. Reporting facts regarding behavior is preferable to the collection of interpretations, ratings and opinions based on general impressions.2. Reporting should be limited to those behaviors which make a significant contribution to the activity • Helps to focus in on key event • e.g. Box 8.5, analyzing video material

24. Tools to support data analysis • Qualitative data analysis tools • Categorization and theme-based analysis • N6 can be used to search a body of text to identify categories or words for content analysis • N6 is used to handle large set of data. • Quantitative analysis of text-based data- Focuses on the number of occurrences of words or words with Similar meanings.

25. CAQDAS CAQDAS Networking Project, based at the University of Surrey (http://caqdas.soc.surrey.ac.uk/) • support theory-building through the visualization of relationships between variables that have been coded in the data. CAQDAS is useful • Grounded Theory • Qualitative Content Analysis • Ethnography

26. SPSS SPSS is Statistical Package for the Social Science • It is used by market researchers, health researchers, survey companies, government, education researchers, marketing organizations and others Statistical packages, e.g. SPSS • You can access, manage, and analyze enormous amounts of data with SPSS. • SPSS offers statistical test for frequency distributions, rank correlations, regression analysis, and cluster analysis

27. Observer Video-Pro • The Observer Video-Pro is a system for collecting, managing, analyzing, and presenting observational data. • It integrates The Observer software with time code and multimedia hardware components.

28. Observer Video-Pro The software allows the user to summarize research findings in numerical, graphical, or multimedia format http://www.noldus.com/human-behavior-research/products/the-observer-xt

29. Theoretical Frameworks for Qualitative Analysis Structuring the analysis of qualitative data around a theoretical framework can lead to additional insights that go beyond the results found from the simple techniques introduced earlier. Three frameworks are discussed in this section: Grounded theory, Distributed Cognition, and Activity theory.

30. Grounded Theory • Grounded theory aims to develop theories from systematic analysis and interpretation of empirical data, i.e. the theory is grounded in the data. • A grounded theory is developed through alternating data collection and data analysis: • Data is first collected and analyzed to identify categories, then that analysis leads to the need for further data collection, which is analyzed, and more data is then collected. • Data collection is driven by the emerging theory. • Approach continues until no further insights emerge and the theory is well developed.

31. Grounded Theory • Analysis is mainly to setup Categories. • Category identification and definition is achieved by “coding” the data, i.e. marking the data up according to the emerging categories. • Coding has three aspects: • Open coding – the process through which categories are discovered in the data. • Axial coding – the process of systematically fleshing out categories and relating them to their subcategories. • Selective coding – the process of refining and integrating categories to form a larger theoretical scheme. Categories are organized around one central category that forms the backbone of the theory.

32. Grounded Theory Researchers are encouraged to draw on their own theoretical backgrounds to help inform the study, as long as they are alert to the possibility of unintentional bias. Emphasizes the important role of empirical data in the derivation of theory.

33. Distributed Cognition The people, environment and artifacts are regarded as one cognitive system Focuses on information propagation and transformation A distributed cognition analysis results in an event-driven description which emphasizes information and its propagation through the cognitive system under study.

34. Distributed Cognition It is recommended to have a deep understanding of the domain under study. Even taking steps to learn “the trade” under study. This could take more time than a research team has available. Alternatively, it is possible to spend a few weeks immersed in the culture and setting of a specific domain to become familiar with it.

35. Distributed Cognition The framework can reveal where information is being ‘distorted’ resulting in poor communication or inefficiency. The framework can show when different technologies and the representations displayed via them are effective at mediating certain work activities and how well they are coordinated.

36. Activity Theory Activity theory (AT) is a product of Soviet psychology that explains human behavior in terms of our practical activity with the world. AT provides a framework that focuses analysis around the concept of an ‘activity’ and helps to identify tensions between the different elements of the system.

37. Activity Theory • AT outlines two key models: • A model that outlines what constitutes an ‘activity’ • A model that outlines the mediating role of artifacts • AT models activities in a hierarchical way. • ‘Activity’ – Provides a minimum meaningful context for understanding the individual actions. • ‘Actions’ – behavior that is characterized by conscious planning. • ‘Operations’ – routinized behaviors that require little conscious attention.

38. Activity Theory Activity is motivated. Actions are to accomplish a goal. Actions involve operations.

39. Activity Theory • AT models artifacts in two ways: • Physical • Abstract • Physical artifacts have physical properties that cause humans to respond to them as direct objects to be acted upon. The object usually embody a set of social practices. • A spoon • Abstract artifacts follow the idea of ‘mediation’. A fundamental characteristic of human development is the change from a direct mode of acting to one that is mediated by something else. • A set of rules or symbols

40. Presenting Your Findings • Only make claims that your data supports. • The best method for presenting your finding depends on the audience, the purpose of the study, and the data gathering and analysis techniques used. • Graphical representations (numbers, tables, graphs, etc.) may be appropriate for presenting your findings. • Other techniques are: • Rigorous notations • Using stories • Summarizing your findings

41. Rigorous Notations UML is an example of rigorous notation because it uses notations that have clear syntax and semantics.

42. Using Stories • Used as a basis for constructing scenarios. • May be employed in three ways: • Stories told by the participants • Stories about the participants • Stories constructed from smaller anecdotes or repeated patterns that are found in the data.

43. Summarizing the findings • Summarizing the data by presenting the headline findings, overviews, and detailed content list. • Numbers and statistical values can be very valuable in a summary. • If you found 800 out of 1000 users preferred design A over design B. This statement would be a quick indication of your findings.

44. Summarizing the findings Activity Consider each of the findings below and the associated summary statement about it. What is correct or incorrect about each finding’s statement?

45. Conclusion The kind of data analysis that can be done depends on the data gathering techniques used. Qualitative and quantitative data may be collected from any of the main data gathering techniques: interviews, questionnaires, and observation. Quantitative data analysis for interaction design usually involves calculating percentages and averages. There are three different kinds of averages: mean, mode and median.

46. Conclusion Graphical representations of quantitative data help in identifying patterns, outliers, and the overall view of the data. Qualitative data analysis may be framed by theories. Three such theories are grounded theory, activity theory, and distributed cognition.