Choosing your research approach

1. Choosing your research approach Research Methods

2. First� Before getting down to questions of research methodology� Some issues people asked about after last week�s session�

3. Dates Dissertation proposal. Week 4: Friday 7th March 2003 by 5.00pm. Dissertation approval and allocation of supervisors. Week 6: Friday 21st March 2003. Dissertation submission. Monday 1st September 2003 by 5.00pm. Coursework for this module. Friday 23th May 2003 by 5.00pm.

4. Ethics Linked to �good practice� more generally. If your research concerns people, you need No harm | Informed consent | Confidentiality/anonymity | Access to data and results. University�s �Good Practice� Website http://www.shef.ac.uk/r/researchoffice/RO/Shef-only/principles.html http://www.shef.ac.uk/r/researchoffice/RO/Shef-only/guidelines.html Any research concerning NHS staff or patients must go to a formal ethics committee. State any ethical issues in your proposal.

5. Coursework 1. A revised version of your initial dissertation proposal, encompassing the preliminary work that you have carried out during Semester 2 (as detailed in the section in the Dissertation Handbook entitled �Initial Progress of Dissertation�). You should include a bibliography of the relevant literature identified in your literature review thus far (not a full literature review). This section should be 2,000-2,500 words long in total, not including the bibliography.

6. Coursework 2. A critical review of a previous dissertation produced by a student on the Masters programmes, on the same (or a closely related) area as your own research project. 1,250-1,500 words long in total, not including the bibliography. In some cases, an externally published report or paper(s) may provide a more appropriate, alternative source for your critique. However, whatever you choose must be agreed with your dissertation supervisor.

7. Coursework Typical criteria: � Aims/objectives/scope. Are these clearly spelt out? Are the research problems and/or rationale well defined? � Research context. Is the project related to previous work? Does it draw on the literature in the field? Is the research or organisational environment described? � Methodology. Is there a full account of the research methods adopted? Are they justified and are the constraints and limitations assessed?

8. Coursework Typical criteria Methodology (continued) In the case of a survey, what is the sample size, how representative is it, is the questionnaire or interview schedule explained? Alternatively, in the case of an application project, is there a justification for the design approach or the tools selected? What constraints affected the design. Was an appropriate evaluation built in to the design?

9. Coursework Typical criteria: � Results and findings. How well are the results presented? Are they appropriately interpreted and discussed? In the case of an application is the final system appropriately documented, and to what extent did the final implementation meet the specification? � Conclusions/recommendations. What are the conclusions? To what extent do they relate to the aims and objectives and are they generally valid? Are the recommendations for future work achievable and prioritised?

10. Coursework Typical criteria: � Presentation. Is the dissertation well presented, clearly written and appropriately structured? Is the abstract informative? Are the references correctly cited?

11. Research Methods & Dissertation Preparation Research �paradigms� abound - e.g. Burrell and Morgan's (1979) functionalist, radical humanist, radical structuralist and interpretative paradigms in information systems; and Olaisen's (1991) empirical, materialistic, action and clarified subjectivity paradigms in information science.

12. Research Methods & Dissertation Preparation But arguably we can think broadly of 2 main approaches: �positivist� and �interpretative�. These are often aligned with �quantitative� and �qualitative� approaches.

13. Traditionally research in the physical sciences has concentrated on... analysing complex situations into component parts, studying them, then reassembling the parts into the original whole with increased understanding.

14. A number of assumptions underlie this approach, namely... we can increase our understanding of complex wholes by analysing them into component parts, better understanding the parts, then re-assembling them to form the whole; we can discover universal laws of behaviour: that is, we can identify variables which when subjected to the same conditions behave in exactly the same way in similar samples.

15. These basic assumptions themselves involve other assumptions, namely... we can measure individual variables in isolation from one another (as opposed to defining and knowing them through their relations with each other); having done so, we can profitably re-link them using statistical relationships; we should control variables and avoid the intrusion of uncontrollable elements.

16. These assumptions require a strong emphasis on... quantitative objectively measurable data; statistical significance testing in order to predict to other samples in a search for universal laws; searching for discrete relationships, particularly cause and effect relationships

17. Limitations� This approach may be highly productive in studying e.g. plant biology. But has limitations when it comes to studying e.g. plant ecology, or the effects of gardens on human moods!

18. �Interpretative� approaches rest on other basic assumptions... the "whole" is more than simply the sum of the parts, and cannot be fully understood by means of isolating, analysing, testing, then reassembling the parts; complex situations must also be studied in their entirety, in a relatively holistic way

19. Interpretative approaches rest on certain basic assumptions... human behaviour is too complex necessarily to allow us to reduce it to universal laws. It is also necessary to study complex interacting phenomena which may interact differently in different contexts; instead of separating and defining variables in isolation from one another, phenomena must be studied within the context of their interactions

20. Interpretative approaches rest on certain basic assumptions... relationships cannot solely be conceived of as discrete e.g. causes and effects rather, relationships must also be seen as complex patterns of mutual interaction findings that relate to restricted contexts, as opposed to those having universal applications, are valid.

21. Interpretative approaches rest on certain basic assumptions... it is possible/desirable to place more emphasis on qualitative data and analysis, even if, as is usually the case, this means reduced quantity (i.e. the number of cases studied).

22. Interpretative approaches Olaisen (1991) considers that this pole of the research paradigms dimension is also characterised by high-complexity problems, an emphasis on social-intuitive as opposed to more logico-mathematical analysis, and "sensitising" as opposed to "definitive" concepts.

23. Interpretative approaches Sensitising concepts Olaisen [1991: 254] are tentative and speculative concepts that: "... offer a general sense of what is relevant and will allow us to approach flexibility in a shifting, empirical world to 'feel out' and 'pick one's way in an unknown terrain. ... In sum, the on-going refinement, formulation, and communication of sensitizing concepts must inevitably be the building block of our exploratory theory."

24. Interpretative approaches Approaches located towards this pole are better able to address problems in the "what we don't know that we don't know" as opposed to the "what we know that we don�t know" category. This arguably entails relatively divergent thought in comparison with so-called "scientific" approaches.

26. Differences Understanding Its nature: what it is you are explaining Its generalisability

27. Differences Evidence Its nature Its qualities validity; reliability; neutrality Procedures for obtaining it sampling; data collection; data analysis

28. Interpretative approaches Relatively holistic understanding of a large number of complex interactions amongst possibly fuzzy aspects of a low number of cases with inclusion of rich context. Generalisable in the sense of representing deep understanding of a complex whole enabling us to understand other cases where similar features and conditions apply. Evidence based on the internal coherence and plausibility (perceived explanatory power) of their findings.

29. Positivist approaches Relatively narrow understanding of the incidence of simple �common denominator� relationships between a small number of well defined isolated variables in large numbers of cases, entailing minimal �extraneous� context. Generalisable in the sense of being statistically predictable to occur in the wider population of which your cases were a representative sample. Evidence based essentially on statistical probabilities.

30. To summarise...

31. Limitations Positivist studies may often suffer from a relative lack of ecological validity, due to the complexity they are often forced to eschew in the interests of experimental control and precise numerical measurability.

32. Limitations Olaisen [1991, p. 260], for example, attributes fragmentation to a paradigmatic imbalance: "... information science thought has been imprisoned by the dominant quantitative empirical metaphors which have drawn the attention to some quantitative phenomena while neglecting other more qualitative phenomena. The result is a cumulation of trivial findings."

33. Limitations Interpretative studies tend to offer a relatively subjective type of evidential support, due often to smaller samples and the preservation of complexity resulting in data which is not particularly susceptible to objective measurement and statistical testing.

34. Limitations Ford [1999, p. 1151] has drawn attention to the dilemma posed by these differences: "Overly-analytic states of knowledge are characterised by fragmentation - at worst, isolated facts lacking integration into any coherent wider conceptual picture... As a result, much research in information science has arguably provided highly reliable answers to highly meaningless questions.

35. Limitations The take-up of qualitative research approaches is now widespread in user-oriented research. But without critical interaction with complementary perspectives the increasing use of subjective analysis of introspections using small samples of information users threatens to supply highly meaningful questions with highly unreliable answers. Some balance and integration must be achieved between the two extremes.�

36. Limitations The limitations associated with research may be thought of as a curtain preventing us from viewing the reality beyond, that we seek to understand. Our existing knowledge ranges between two extremes, which to some extent mirror the different research approaches discussed above.

37. Limitations One may be characterised as scattered pinpricks in the curtain, allowing clear and deep, but narrow and unconnected views through to the reality beyond. The other may be characterised as more extensive areas where the curtain is thinned, allowing complex, inter-connected but hazy shapes to show through, inviting us to trace them onto the curtain, elaborating their detail to represent what we imagine to be their reality.

38. Limitations It is arguably all too easy to ignore the extent of curtain and our consequent lack of clear view, due to a variety of (sometimes unconscious) forms of over-optimism as to how securely and how widely we can generalise research findings to form a reliable and relevant picture.

39. Forms of over-optimism include falsely equating constrained research contexts with more complex reality is a form of over-generalisation most often associated with quantitative research in which experimental control and accurate measurement of variables are paramount.

40. The danger is in... assuming that what is observed in experimental conditions also applies in non-experimental, more natural conditions (much positivist research); and/or confusing internal coherence and plausibility with generalisability (much interpretative research).

41. Different views... Sometimes a broad holistic view can paint a picture...

43. Different views... Sometimes a broad holistic view can paint a picture... that does not stand up to more precise objective measurement and calculation

47. Different views... Sometimes a narrow analytic view of parts of a whole paint a clear and logical picture...

49. Different views... Sometimes a narrow analytic view of parts of a whole paint a clear and logical picture... that requires a different interpretation when a broader more holistic view is taken

53. Problems Sometimes it is difficult to see things from different perspectives at the same time...

59. Problems As Marton and Svensson [1979, p. 484] note in relation to research approaches: "What we can see from one point of view we cannot see from another. . . . With one kind of observation certain aspects become visible: with another kind of observation we see something else. We cannot arrive at a procedure of observation which makes all the various aspects visible simultaneously."

60. Paradigm wars Entwistle and Hounsell [1979: 363] note that: "As each paradigm marks out boundaries and establishes its own rules of discourse, there is a danger that territorial advantage will be sought through confrontation rather than mutual understanding-and the outcome of a pitched battle is more likely to be schism than synthesis."

61. Paradigm wars Burrell and Morgan [1979: 397-8] propose that, in the interests of self-preservation and to avoid emasculation: "Contrary to the widely held belief that synthesis and mediation between paradigms is what is required, we argue that the real need is for paradigmatic closure."

62. Paradigm wars Bradley and Sutton [1993: 407] - are of the opinion that the conflicts are artificial: "The paradigm debate has, in some senses, created an artificial polarisation based on abstractions that can easily harden into misunderstanding, caricature, and an attitude of superiority on both sides."

63. Integration of approaches Entwistle and Hounsell [1979, p. 361] note: "The two paradigms ... contain the tension of opposites - a thesis and antithesis out of which a fruitful synthesis might be anticipated, but is still far from being achieved ... Yet the methodologies of competing paradigms could be used alongside one another, each providing distinctive yet equally valid types of evidence."

64. Complementary strengths

65. Complementary strengths

66. Strategies for methodological pluralism include: the mapping of different paradigms onto different kinds of problem; the use of different paradigms within a single study; critical dialogue relating to a common phenomenon from different paradigmatic perspectives.

67. Vertical & horizontal strength Commitment to particular research approaches may often centre on decisions as to how deep is "deep enough". As Popper [1968, p. 111] noted: "... if we stop driving the piles deeper, it is not because we have reached firm ground. We simply stop when we are satisfied that the piles are firm enough to carry the structure, at least for the time being."

68. Triangulation For some phenomena, combining narrow analytic and broad holistic perspectives may enhance and strengthen understanding like shining lights on the same phenomenon from different angles to give a clearer picture

69. Triangulation Other phenomena, may only be truly understandable from one perspective � and not be susceptible to the triangulation of approaches. Alternatively, different approaches may lead to very different types of understanding.

70. References Bradley, J. & Sutton, B. (1993). Reframing the paradigm debate. Library Quarterly, 63 (4), 405-409. Burrell, G. & Morgan, G. (1979). Sociological paradigms and organisational analysis. London: Heinemann. Ellis, D. (1996). The dilemma of measurement in information retrieval research. Journal of the American Society for Information Science, 47 (1), 23-36. Entwistle, N.J. & Hounsell, D. (1979). Student learning in its natural setting. Higher Education, 8, 359-363. Ford, N. The growth of understanding in information science: towards a developmental model. Journal of the American Society for Information Science, 50(12), 1999, 1141-1152.

71. References Marton, F. & Svensson, L. (1979). Conceptions of research in student learning. Higher Education, 8, 471-486. Olaisen, J. (1991). Pluralism or positivistic trivialism: important trends in contemporary philosophy of science. In H.E. Nissen, H.K. Klein & R. Hirschheim (Eds.). Information systems research: contemporary approaches and emergent traditions. Amsterdam: Elsevier. pp. 235-265. Popper, K. (1968) The logic of scientific discovery. New York: Harper and Row.