OMIS 694 Advanced ISD Topics

1. OMIS 694Advanced ISD Topics Fall 2002

2. What is research? �the process of systematically obtaining accurate answers to significant and pertinent questions by the use of the scientific method of gathering and interpreting information� Balsley & Clover, in Research for Business Decisions

3. The Scientific Method Become aware that a problem exists Define the problem Set forth hypotheses as to cause and/or solutions Determine what information will be required Decide which methods will be used in collecting information Collect information or evidence Compile findings in systematic form Analyze findings to determine whether they substantiate or eliminate hypotheses Write the final research report

4. Alternative: What is research? �activities undertaken to increase knowledge in an area of study� anonymous

5. Why do we do research? To get information To gain understanding To answer a question To assist in evaluating options To predict outcomes

6. How do we do research? Select & define a problem research question purpose of the research Execute the research procedures methodology data collection subjects Analyze the data Draw & state conclusions

7. Research Methodologies Case Study Field Study Field Experiment Laboratory Experiment Conceptual/Normative Research Others Basic & Applied Research Surveys Action Research

8. Case Study Intensive evaluation of small sample of entitles such as groups, organizations, or systems Purpose: explanation, description, hypothesis generation Multiple means of data collection No controls of experimental phenomena

9. Field Study An ex post facto method of evaluating organizational systems Purpose: Description, hypothesis generation or hypothesis testing Data Collection: questionnaires, interviews No experimental variables are manipulated

10. Field Experiment Experimental manipulation of one or more independent variables while confounding variables are controlled Purpose: To observe the effect of the independent variable(s) in a natural setting, typically evaluated by hypothesis testing Many types of data collection possible

11. Lab Experiment Research takes place in a researcher-created setting with researcher manipulation of independent variable(s) and a high degree of control Purpose: Hypothesis testing Many types of data collection possible

12. Conceptual/Normative Non-empirical (no data) research Based on experience, opinion, or synthesis of existing literature Examples Frameworks Description of an organization�s experience with a particular system

13. Goals of Research Design Right issues Correct approach methodology experimental design subject/site/task/measures Reliability & validity of results Valid conclusions

14. Steps in Designing Research Study Research question Hypotheses Variables Methodology Data Collection Methods who? how? Data Analysis Methods

15. Writing a Research Question Is it meaningful? Is it understandable? Is it researchable? Does the question become moot if you get opposite results?

16. Developing Research Hypotheses An hypothesis is a tentative explanation for certain behaviors, phenomena, or events that have occurred or will occur The researcher does not set out to �prove� the hypothesis but rather collects data that support the hypothesis Hypotheses should be based on theory, previous research, or observation of actual events

17. Developing Research Hypotheses Hypotheses should state as clearly and concisely as possible the expected relationship (or difference) between two variables in operational, measurable terms Hypotheses must be testable

18. Developing Research Hypotheses A general model for stating hypotheses is: S�s who get X do better on Y than S�s who don�t get X (or get some other X) Research hypotheses are stated in declarative form (directional or non-directional) Statistical hypotheses are stated in null form (no difference)

19. Research Constructs vs. Variables Constructs are the basic concepts that we are examining user satisfaction decision making software quality Variables are the measurable values that represent the constructs

20. Types of Variables Dependent variables Independent variables experimental: manipulated by researcher quantitative qualitative Extraneous variables any factors other than those under study that could have an affect on the dependent variable

21. Types of Variables Intervening VariableInd. Var. --> Inter. Var. --> Dep. Var. Moderating VariableInd. Var. ------------------> Dep. Var. Mod. Var.

22. Measurement Instruments Using existing vs. developing new Methods of measurement Closed vs. open ended questions Ranking & comparing Scales

23. Scales nominal ordinal interval ratio Likert semantic-differential

24. Selection of Sample/Subjects Steps Define population Determine sample size Select sample Importance of sample selection Error vs. bias

25. Sampling Techniques Random sampling Stratified random sampling Cluster sampling Systematic sampling Convenience sampling Judgmental sampling Quota sampling

26. Validity Internal Validity External Validity Statistical conclusion Validity Construct Validity Face Validity

27. Internal Validity Threats history maturation sensitization instrumentation statistical regression selection mortality

28. External Validity Threats selection of sample selection of setting selection of tasks interaction of treatments

29. Statistical Conclusion Validity Threats small sample size measures with low reliability

30. Construct Validity Threats too few measures wrong or poorly defined measures

31. Face Validity Threats results contrary to theory or norms low credibility of methods

32. Statistical Power Statistical Power = the probability that a statistical test will correctly reject a null hypothesisHo No differenceHa Difference

34. Statistical Power Determinants of Power alpha sample size effect size Increasing Statistical Power increasing sample size using more powerful statistical tests random sampling MAXMINCON No significance does not equal no effect

35. MAX MIN CON MAXIMIZE differences between the treatments (effect size) MINIMIZE error (quality control) CONTROL extraneous variation