slide1 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Student readiness PowerPoint Presentation
Download Presentation
Student readiness

Loading in 2 Seconds...

play fullscreen
1 / 26

Student readiness - PowerPoint PPT Presentation

  • Uploaded on

Student readiness. Supervision challenges. Towards a Thesis Assessment Matrix: An action research project Michelle Picard & Lalitha Velautham. Challenges of format. New examiner challenges. Literature on thesis examination.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Student readiness' - heman

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Student readiness

Supervision challenges

Towards a Thesis

Assessment Matrix: An

action research project

Michelle Picard & LalithaVelautham

Challenges of format

New examiner challenges

literature on thesis examination
Literature on thesis examination
  • ‘There seems to be a tacit understanding of what constitutes a well-prepared Ph.D. student’, and that ‘in the complete absence any central repository of rules or a cosmic accrediting agency’ there is ‘extraordinary stability’ (Kwiram, 2006 p. 142 cited in Holbrook et al, 2008).
  • Experienced examiners generally give feedback consistently. Inexperienced examiners find it difficult to define acceptable thesis standards and are inconsistent in giving feedback (Kiley & Mullins, 2004; Kiley et al, 2011).
  • Need for indicators to distinguish between theses of threshold quality and those of highest quality (Holbrook et al, 2008)
  • The global research/ the global thesis standard?

RED, School of Education

literature on assessment matrices
Literature on assessment matrices
  • Tools to negotiate expectations between supervisors and students (Cadman & Cargill, 2007, Kumar & Strache, 2007)
  • Taxonomies for reviewing literature (Cooper, 1988; Boote and Beile, 2005; Randolph, 2009)
  • Thesis rating scale (Albertyn et al, 2011)
areas for extension
Areas for extension
  • No clear empirical basis
  • Lack descriptive language and quality indicators commonly employed by experienced examiners
  • Disciplinarily slanted towards Sciences
  • Focussed on traditional thesis formats
research aim
Research aim
  • To create thesis assessment discussion tools for supervisors, students and (potentially) examiners in order to clarify expectations and assist in assessing research documents across formats and disciplines

RED, School of Education

theoretical framework and methods
Theoretical Framework and Methods

Identification of issue

Development of theory

Development of matrices

Deployment & evaluation of matrices

(Drummond & Themessl-Huber 2007)

RED, School of Education

action research spiral 1
Action research spiral 1

1. Research proposal assessment matrix

  • Developed and refined based on discussions with EAL students and their supervisors
  • Final product developed from RSD7 & Literature Review (Willison, 2008 & Boote & Beile 2011)
  • Used with over 600 research students from March 2010 and their supervisors as:
  • Negotiation tool
  • Assessment tool
  • Overwhelmingly positive evaluation
  • Wordiness

RED, School of Education

action research spiral 2
Action research spiral 2

2. First draft thesis assessment matrix

  • Imperatives from our experience & the literature as well as work on Research Proposal Matrix
  • Draft matrix developed from RSD7 & Literature Review (e.g. Willison, 2009; Randolph, 2009; & Boote & Beile 2011) as well as insights from spiral 1.
  • Deployment to experienced supervisors at the University of Adelaide via an online survey, by students in a thesis writing group and at a postgraduate research conference in South Africa (April, 2013).

RED, School of Education

student comments spiral 2
Student comments Spiral 2

HDR students

  • ‘It is good to have something to take in to discuss my writing with my supervisor’ (HDR 15).
  • ‘It helps me to know what I must do to do research in a PhD’
  • ‘It is scary to see how much I still have to learn, I thought I knew all about research in honours’ (HDR1
  • ‘It is a relief to see I am ok, at a stage of development just like my supervisor’ (HDR 21)
  • ‘This grid really helped us to see what is expected of a know you say it is not a Nobel Prize, but sometimes my supervisors seem to expect that... so it helps to know what a thesis really should do’.
  • Abit ‘wordy’ (HDRs 1,12 & 16)
conference comments
Conference comments
  • ‘Too focussed on literature review and writing skills’
  • ‘Needs more focus on original contribution’
  • ‘Good to consider thesis by publication’
  • ‘Likely to be less useful in creative disciplines’
  • ‘Does not really address argument-based thesis’
  • ‘It still does not encapsulate the true work of the PhD’
positive responses in
Positive responses in….
  • Clarify expectations for research students
  • Clarifying expectations for new supervisors
  • Clarifying expectations for new examiners (but also high strongly disagree)
  • Aid in researcher training
positive comments
Positive comments
  • ...this would be very helpful for examiners - provides clear differentiation and progression, and what is suitable for each level.(ID 12)
  • ...any template which gives students, supervisors, etc. an understanding of levels of competency will assist them to distinguish between competent and excellent work. (ID 12)
  • the criteria listed coincide with the long established measures of adequacy in our discipline. (ID 17)
less favourable responses in
Less favourable responses in…
  • Assist new examiners (22.2 strongly disagree)
  • Other thesis formats (27.8 disagree)
  • Differentiate between MPhil, Doctorate, PhD (33.3 neutral)
examiners supervisors
Examiners & Supervisors
  • introduces thresholds that do not correspond with the Pass/Fail approach to research degree examination. There is absolutely no point in differentiating between 'Pass' and 'Excellence' at the PhD level. (ID 10)
  • New students, supervisors and even examiners may still lack the knowledge or confidence, even with these criteria, to work out where their work sits... (ID 12)
  • If a supervisor doesn't know what is entailed in a thesis of particular kind at a particular level, he/she should not be a supervisor. (ID 16)
formats disciplines
Formats & disciplines
  • Please note that creative theses (creative writing PhD and MPhil) would need an adapted assessment tool - to be developed by the relevant discipline(s). One size does not fill all.(ID 3)
  • fails to address metrics related to thesis writing specifically in the Sciences related to areas of poor experimental design, data analysis, and data interpretation
  • ‘It wasn't clear to me that the draft assessment tool provided these metrics, but rather was couching inadequacies within more general and rather heavily semantic terminologies - the RSD has its own lingua franca that may not be widely understood or appreciated, and certainly I'd suggest is beyond the interpretation of a busy international examiner. (ID 8)
  • ‘Experienced examiners will dismiss it. Inexperienced examiners will be misled by it.’ (ID 10)
  • ‘ is implicitly language rich, and with 50% of our students coming to UA from o/seas, to introduce them to such refinement of language early in their research careers, is surely asking a bit much.(ID 8)
  • If it is to have 'currency' it would need to have greater clarity through simplicity. The essential feature for a PhD, for example, is originality. This is the crucial ingredient. For the MPhil, however, the originality looms less large. (ID 10)
  • ‘excellent research outcomes’ often overshadows ‘poor thesis construction’. (ID 8)
  • A PhD degree is examined on a Pass/Fail basis. There is no recognised international concept of a 'Pass' PhD as opposed to a 'Better than Pass' PhD. (ID 10)
action research spiral 3
Action research spiral 3

3. Refinement of thesis assessment matrix

  • Audit of rubrics provided to Australian and international thesis examiners by institutions
  • An audit of recurring phrases indicating quality and rigor in examiners reports
  • Combination of all data into revised thesis assessment matrix
  • Thus far, audit of rubrics completed

RED, School of Education

rubrics across go8 universities
Rubrics Across GO8 Universities
  • Knowledge contribution PhD

“Thesis as a whole is substantial and original contribution to knowledge of the subject…” (ANU; UWA)

“Significant contribution to knowledge and understanding in the relevant field” (MU)

“Makes a distinct contribution to knowledge” (UOS)

“Thesis displays new perspectives and/or advanced knowledge…” (UNSW)

“makes a distinct contribution to knowledge because of the originality of the approach and/or interpretation of the findings and in some cases the discovery of new facts” (UM)

University of Adelaide

rubrics across go8
Rubrics Across GO8
  • Knowledge contribution Masters

“Shows a sound knowledge of the subject…evidence of some independence of thought” (UWA)

“Makes a contribution…” (MU)

“Appreciation of relationship of topic to wider field of knowledge” (UOS)

“Displays new perspectives” (UNSW)

“Demonstrates advanced learning in research skills” (UM)

potential matrix advantages
Potential matrix advantages
  • Supervisors: Clear articulation of expectations and standards
  • Examiners: Fair, transparent and consistent evaluation, assistance in articulating concerns
  • Students: Clear goals and direction leading to better self-management techniques in thesis writing

RED, School of Education

a word of caution
A word of caution
  • Hijacking by rampant managerialists/ administrators
  • Difficulties of pinning down the ephemeral
  • Danger of ‘marking’ rather than ‘examining’ (Ward, 2013).
  • Matrix/grid implies grades – the PhD is not graded per se
  • Disciplinary peer review element of becoming ‘part of the club’, entering into an in-depth discussion could potentially be lost
  • Personally now with two PhD examinations – still need for assessment and comments, but helped in examining process
next step in action research cycle
Next step in Action Research Cycle
  • Audit of reoccurring phrases/ language used by experienced examiners
  • Refining current matrix (in handout)
  • Trial of current matrix with examiner volunteers
questions feedback
Questions & Feedback
  • Please contact:


RED, School of Education