Writing up the qualitative report & assessment criteria

Writing up the qualitative report & assessment criteria Colm Crowley University of Greenwich

Anything goes? • Given the great diversity of structure and style in published qualitative research reports, how should we guide our students? • While giving students a free hand can be a valuable part of their induction into qualitative methods at postgraduate level, it is more likely handicap them at first degree level: • For those new to qualitative methods, a free hand in report writing style can increase their bewilderment

Strategy - Using common principles to increase confidence • Build on the principles and formats common to both quantitative and qualitative report writing • Highlight differences needed in qualitative research reports

Keeping it simple for 2nd year project students • Be quite prescriptive • Use modified scientific report format as framework • But let students know that this is for pragmatic reasons – tried and tested strategies rather than rigid principles • Asking ‘ How effectively will this convey what readers will want to know?’ and ‘How could it be improved further?’ rather than ‘Is this right or wrong?’

Modified scientific report structure • Title • Abstract • Introduction • Methodology • Analysis (or Findings and discussion) • Conclusion • References • Appendices

Highlighting key differences in qualitative reports • Differences in emphasis (compared with quant reports) -- Title -- Introduction -- Methodology -- Appendices • Differences in structure (compared with quant reports) -- Findings -- Discussion

Title • Because it still tends to be an unspoken assumption in much of psychology that methodology will be quantitative, this is not usually mentioned in a title (although it can perhaps be deduced from some titles). • Likewise, it can be apparent from titles that contain words (such as ‘narratives’, ‘accounts’, ‘personal meanings’, ‘lifeworlds’, ‘discourses’, ‘views and experiences’, ‘social/media representations’, ‘social construction’, ‘positioning’, etc.) that the research reported will be qualitative.

Title • However, it is good practice to explicitly indicate in a title that a qualitative methodology is used, as that might be of particular interest (or disinterest) to potential readers. It also makes the task of literature searching by methodology more efficient. • While keeping the title as short as possible, inclusion of phrases such as ‘a qualitative/ ‘a phenomenological’/ ‘a grounded theory’/ ‘a conversation-analytic’ investigation/study/inquiry’ in the title (or subtitle) is regarded as helpful information.

Writing style – breaking ‘the rules’? • The writing style of a qualitative report is generally the same as with quantitative reports and other academic work (i.e., writing in an indirect style or reporting in the past tense). • But writing in a more personal way, using ‘I’, is congruent with owning one’s position • So it is advisable to use first person forms where appropriate and (in student work) to state one’s intention to do this (e.g., when discussing reflexivity).

Introduction – the literature • The literature review, while giving some background on the overall literature relevant to the topic, should ideally identify which studies reviewed are quantitative and which qualitative. • When the review focuses in more depth on qualitative findings in the literature, it will assist in the subsequent discussion of the analysis. • So it is worthwhile to search specifically for relevant qualitative literature (PsycInfo’s quant/qual methodology search filter menu can be useful).

Introduction – aim • Having summarised the rationale (based on the literature), the research aim can be explicitly stated. • A fairly broad aim is good for an exploratory study. For example, ‘… to gain a greater understanding of the personal meanings of xxxx to xxxx’. ‘… to gain a greater understanding of the lifeworlds of xxxx’. • Tip: the stated aim of a piece of qualitative research could echo the report’s title somewhat (in a reworded form) because it is likely that the scope of both (the focus and what is specified) would coincide.

Introduction – research question/s • Hypotheses usually not appropriate, certainly if the research aims to explore rather than verify (distinction between ‘Big Q’ and ‘small q’ qualitative methodology – see Kidder & Fine, 1997) • For example ‘My initial research question was: What are the views and experiences of the participants regarding xxxx? A further research question was: What gender (cultural?) differences are there in the views and experiences of participants?’

Useful sub-sections for Methodology • Design • Participants • Methodological theory • Method of data collection • Procedure • Ethical considerations • Analytic strategy • Reflexivity • (but generally not Materials/ Apparatus)

For a conversation analysis report, Methodology sub-sections might be • Design • Participants, material and context • Procedure • Reflexive account of selection criteria • Transcription

Design sub-section • This sub-section can be brief. The main issue is that of the design being flexible rather than fixed (see Robson, 2002) rather than just ‘a qualitative design’, together with which methodology/ies are appropriate (echoing the rationale in the Introduction) . • For example, if using a single-case design (perhaps in an IPA study) the work of Flyvbjerg (2006) might be referenced. • Similarly, if using a participant-observational design, the work of Flick (1998) might be referenced.

Methodological theory sub-section • This might be GT, CA, phenomenology, or DA, for example. One of these fairly broad categories of qualitative methodology will have been mentioned in the Design sub-section. However, each tends to have some differing forms, with each form having its advocates and critics in the literature. • So the task of this subsection is to identify which particular form of the overall methodology was used (and perhaps why you consider this to be more appropriate than alternative forms) and to outline its distinguishing features (supported by referencing)

Analytic strategy sub-section • This sub-section will relate to the methodological theory described in the earlier sub-section (but be careful to avoid any overlap). • It should succinctly explain the steps carried out in data analysis, beginning with the approach taken to transcription and the conventions used for it, e.g. ‘as detailed by Forrester (in press)’. • For example, ‘themes were identified in the following way … using procedures for IPA analysis described by Shaw (in press)’.

Reflexivity sub-section • This is increasingly considered to be a criterion for the evaluation of qualitative research – perhaps a qualitative parallel to quantitative considerations of validity and reliability, and seen to be a means of increasing the transparency of the research process. • This sub-section should therefore demonstrate an understanding of the role of reflexivity in qualitative research (again with references such as Willig, 2008) and explain the procedures for it in the present study.

Reflexivity sub-section • For example, the systematic keeping of memos regarding the steps of data collection and analysis should be reflexive and not just descriptive. Providing these in appendices could constitute an important illustration of how reflexivity was practiced. • Providing a ‘personal statement’, or ‘reflexive account’ is also often recommended, and this sub-section could indicate where this can be found in the report (most likely in an appendix due to word-limits).

Analysis • In a qualitative research report, in order for the section following Methodology to make sense, it needs to contain a good deal of discussion. This is certainly the main presentational difference between quantitative and qualitative methodological reports. • So it is more likely to be called Analysis or Findings (or Findings and discussion) than Results. • The Analysis section will also reflect the kind of qualitative method used – both in terms of how this section might be structured and how the content is described

Analysis • For example, when presenting the quotations providing examples of the themes, it is necessary to introduce and discuss them to some extent. • As well as this sort of contextual discussion (considered to be an element of the analysis in qualitative research), it is also necessary to consider the findings in the light of the literature reviewed in the introduction section. • It can be more efficient to do this as you go along than to put it in a subsequent discussion section.

Conclusion • However, other, more general issues need to be discussed by way of conclusion anyway - e.g. overall conclusions (interpretation of findings) from the Analysis section (drawing out the main points in the light of the research questions) • This should relate the overall findings to the literature reviewed, whereas in the Analysis section more specific issues would have been discussed in the light of the literature.

Appendices • Making good use of an appendix is a major element of a qualitative report – so it can get very big! • When students collect their own data, as with quantitative projects, include: ethics paperwork, consent form, recruiting and ‘debriefing’ information. • When relevant to the methodology, a personal reflexive statement is often best included in an appendix (to get around word limits). • For CA and DA, notes about or samples of specific material or procedures involved in the research, extract transcripts.

Appendices • For thematic analysis (GT or IPA), at least one full interview transcript with (handwritten) initial coding on it (e.g. line-by-line coding, open coding). • Also, for the thematic forms of analysis: comprehensive lists of codes/themes/categories. • Include all initial lists and revised lists, plus all other lists produced in the development of the analysis; examples of memos; examples of theme/category descriptions or definitions (if not already given in Analysis section); diagrams (or lists or tables) of clustered themes from which main themes were developed.

Assessment – discussion points • Comparison of marking frameworks for a quantitative and a qualitative 2nd year lab report • Note: In developing this example of a qualitative marking framework, the quantitative marking framework was adapted only where needed (rather than rewriting it fundamentally). • The final (qualitative) comments box can highlight some limitations & strengths. • ‘Interactive feedback’? But... time issues.

References • Flick, U. (1998). An introduction to qualitative research. London: Sage. • Flyvbjerg, B. (2006) five misunderstandings about case-study research. Qualitative Inquiry, 12, 219-245. • Kidder, L.H., & Fine, M. (1997). Qualitative enquiry in psychology: A radical tradition. In D. Fox & I. Prillentensky (Eds.) Critical psychology: an introduction. London: Sage. • Robson, C. (2002). Real world research (2nd ed.). Oxford: Blackwell. • Willig, C. (2008). Introducing qualitative research in psychology (2nd ed). Buckingham: Open University Press.

Writing up the qualitative report & assessment criteria