1 / 23

Methodology and Explanation XX50125

Methodology and Explanation XX50125. Lecture 1: Part I. Introduction to Evaluation Methods Part 2. Experiments Dr. Dana ë Stanton Fraser. Goals of this unit. To introduce research methods in human computer interaction To explore qualitative and quantative methods

tate-ball
Download Presentation

Methodology and Explanation XX50125

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Methodology and ExplanationXX50125 Lecture 1: Part I. Introduction to Evaluation Methods Part 2. Experiments Dr. Danaë Stanton Fraser

  2. Methodology and Explanation 2007 Goals of this unit • To introduce research methods in human computer interaction • To explore qualitative and quantative methods • To gain experience in putting these methods into practice • To gain a critical understanding of analytic techniques

  3. Methodology and Explanation 2007 Unit structure Lectures each focusing on methods of analysis Practical sessions putting these methods into practice

  4. Methodology and Explanation 2007 Course text and slides Rogers, Y., Sharp, H., and Preece, J. 2007. Interaction Design: Beyond Human Computer Interaction. Wiley. Slides available online at: http://staff.bath.ac.uk/pssds/M&E.html

  5. Methodology and Explanation 2007 Assessment 2 assessed courseworks: 1st coursework (report) 2nd coursework (report and presentation)

  6. Methodology and Explanation 2007 Evaluation Methods • Controlled experiments • Ethnography • Expert review • Usability testing • Surveys/questionnaires • Data logging • Interviews and focus groups

  7. Methodology and Explanation 2007 Not just desktop computing. What other kinds of technologies are we evaluating?

  8. Methodology and Explanation 2007 Direct manipulation & graphical user interfaces

  9. Methodology and Explanation 2007 Collaborative Desktop

  10. Methodology and Explanation 2007 Immersive technologies

  11. Methodology and Explanation 2007 Tangible interfaces

  12. Methodology and Explanation 2007 Wearable, handheld and mobile devices

  13. Methodology and Explanation 2007 Embedded interfaces

  14. Methodology and Explanation 2007 Evaluate: which method to choose • Design or implementation? • Laboratory or field studies? • Subjective or objective? • Qualitative or quantitative? • Information provided? • Immediacy of response? • Intrusiveness? • Resources?

  15. Methodology and Explanation 2007 Part II Experiments: A brief guide

  16. Methodology and Explanation 2007 Aim • To answer a question or test an hypothesis that predicts a relationship between two or more events, known as variables. • E.g. Will spatial knowledge be superior following exploration of a VR simulation of a building or from exploring a model of a building?

  17. Methodology and Explanation 2007 Variables • Such hypotheses are tested by manipulating one or more of the variables. • The variable that is manipulated is called the independent variable (the conditions to test this variable are setup independently before the experiment starts). In the study mentioned media type (VR vs model) is the independent measure • The dependent variable would be accuracy of spatial information (ie time to reach point a from point b) because the hypothesis is that the time to carry out the task depends on the media explored.

  18. Methodology and Explanation 2007 Variables and Conditions • In order to test an hypothesis the experimenter sets up the experimental conditions e.g. • Condition 1 Pretest Explore model Post test • Condition 2 Pretest Explore VR Post test • Control condition – against which to compare the results?

  19. Methodology and Explanation 2007 Participants • Between participants Two drawbacks – no. of participants needed, individual differences Advantage – no order effects • Within participants Counterbalancing required • Matched pairs on characteristics such as gender and expertise Difficult to match across all variables

  20. Methodology and Explanation 2007 Practicalities • Where will the experiment be carried out? • How will the equipment be set-up? • How does one introduce participants to study? • What scripts are need to standardise the procedure? • Always include pilot study

  21. Methodology and Explanation 2007 Data Collection and Analysis • Data – performance measures are taken e.g. response times, no. of errors • Use graphs • The data should be averaged across conditions to examine any differences • Statistical tests such as t-tests and ANOVAs can reveal whether the differences are significant • Software packages such as SPSS are often used • If there is no significant difference then the hypothesis is refuted.

  22. Methodology and Explanation 2007 Data captured • Record quantitative results • However also gather process data e.g. dialogue turns, gesture and non verbal behaviour

  23. Methodology and Explanation 2007 Preparation for Practical 1 Topic: Examining the use of gestures in collaboration. In preparation for the practical please read: • Kraut, R. E., Fussell, S. R., and Siegel, J. (2003). Visual Information as a Conversational Resource in Collaborative Physical Tasks. Human-Computer Interaction,Vol. 18, pp. 13–49 Lawrence Erlbaum Associates, Inc. http://www.columbia.edu/itc/psychology/rmk/Readings/Kraut.pdf 2. Kirk, D. and Stanton Fraser, D. (2005). The Effects of Remote Gesturing on Distance Instruction. Computer Supported Collaborative Learning (CSCL 2005). Taiwan. May.

More Related