Developing and assessing the research skills of students in engineering courses
Sponsored Links
This presentation is the property of its rightful owner.
1 / 28

Developing and Assessing the Research Skills of Students in Engineering Courses PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Developing and Assessing the Research Skills of Students in Engineering Courses. Dr Said Al-Sarawi Research Skill Development and Assessment ALTC Project Member School of Electrical and Electronic Engineering University of Adelaide A seminar for Curtin Engineering Faculty. Outline.

Download Presentation

Developing and Assessing the Research Skills of Students in Engineering Courses

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Developing and Assessing the Research Skills of Students in Engineering Courses

Dr Said Al-Sarawi

Research Skill Development and Assessment

ALTC Project Member

School of Electrical and Electronic Engineering

University of Adelaide

A seminar for Curtin Engineering Faculty


  • Motivation

  • Students and staff issues

  • RSD Framework in Australia

  • Benefits of using RSDF

  • What is RSD Framework?

  • Case Studies – From 1st Year to Master Level

  • Other Dimensions of RSD Framework

  • RSD at the Program Level !

  • RSD in conclusion

  • Discussion and future work

Motivation (1/2)

  • PhD completion rate were doubled for students who had participated in undergraduate research (Bauer & Bennett, 2003)

  • Performance based research funding (old RQF, ERA)

  • Performance based learning and teaching support – Learning and Teaching Performance Fund (DEST, 2006)

  • Undergraduate research has been associated with higher level of student satisfaction and generic skills development (Kardash, 2000)

  • Factors affecting skills implementation are (Lucas et al, 2000)

    • Scepticism of the message, the messenger and its vocabulary

    • The skills demanded lack clarity, consistency and recognisable theoretical base

    • The skills are dependent on discipline area

Motivation: Student and Staff issues (2/2)

RSDF in Australia

  • The University of Adelaide (John Willison)

    • Human Biology (Eleanor Peirce & Mario Ricci)

    • Electrical Engineering Masters by Coursework (Said Al-Sarawi and Brian Ng)

    • Clinical Nursing (Frank Donnelly)

    • Petroleum Engineering (Steve Begg)

    • Introductory Academic Program (Richard Warner)

    • English (Joy McEntee)

    • Dentistry (Vicki Skinner and Leonard Crocombe)

    • Oral Health (Sophie Karanicolas and Cathy Snelling)

    • Software Engineering (Li Jiang)

    • Veterinary Science (Susan Hazel)

RSDF in Australia (Conts.)

  • Macquarie University (Psychology: Judi Homewood)

  • Monash University (Business Ethics: Jan Schapper; Sue Mayson: Business; Glen Croy: Tourism)

  • University of Melbourne (Business Law: Eu-Jin Teo)

  • University of South Australia (Introduction to Tertiary Learning, 2008: Rowena Harper)

The facets of student research

In researching, students:

  • embark on an inquiry and so determine a need for knowledge/understanding

  • find/generate needed information using appropriate methodology

  • critically evaluate information/data and the process to find/generate

  • organise information collected/generated

  • synthesise and analyse and apply new knowledge

  • communicate knowledge and the processes used to generate it, with an awareness of ethical, social and cultural issues.

    (Willison & O’Regan, 2007)

Learning to Frame Research Questions

Level 1

Facet A) Embark on Inquiry

Respond to questions / tasks arising explicitly from a closed inquiry.

Facet E) Synthesis, analysis, application Ask questions of clarification / curiosity.

Learning to Frame Research Questions

Level 2

Facet A) Embark on Inquiry

Respond to questions / tasks required by and implicit in a closed inquiry.

Facet E) Synthesis, analysis, application Ask relevant, researchable questions.

Learning to Frame Research Questions

Level 3

Facet A) Embark on Inquiry

Respond to questions / tasks generated from a closed inquiry.

Facet E) Synthesis, analysis, application

Ask rigorous, researchable questions based on new understandings.

Why use RSD approaches? (1/2)

  • Benefits of using the RSD for students (according to Eleanor Peirce and Mario Ricci, Medical Sciences, Uni of Adelaide)

  • Their research skills in our course have improved.

  • They understand much more clearly what is expected of them.

  • They know exactly where they need to develop, thanks to feedback.

Benefits for Lecturers (2/2)

(according to Eleanor Peirce and Mario Ricci, Medical Sciences, Uni of Adelaide)

  • We can give feedback on assessment tasks more accurately and efficiently; we can give the same quality feedback with less writing, and faster.

  • We can get a much better idea of where our students are from a quick analysis of the RSD results than from a detailed analysis of standard marks.

  • We can easily match assessment tasks with course objectives, and course objectives with the University’s Graduate Attributes.

Case Study 1: Human Biology - 1st Year

  • Read the two short then complete tasks 1 and 2.

  • Task 1:

  • Integrate the information presented in the two articles to write your own dot-point notes

  • on the worksheet attached. To do this:

    • Identify 3-4 key ideas from the articles

    • Use these key ideas to formulate headings and underline each

    • Make bullet-point notes and list them under these headings.

    • After each point, indicate its source, i.e. whether the idea came from article 1, article 2, or both

    • Provide a title that embodies the content of your notes.


Which of the two articles do you consider to be the better source? On what characteristics/features of the article have you based your choice?

Case Study 1: Human Biology - 1st Year

Case Study 1: Human Biology - 1st Year

Literature Research Skill Stream

Laboratory Research Skill Stream

Field and literature research

Semester 2

Semester 1

Novelty in the engineering case

The differentiation points:

  • Clearly state the need to research skill development

  • The students are required to identify a gap in knowledge for each of the chosen topics (the unknown)

  • Rigorous literature research

Case study 2: Photonic and Communication – Master Course

  • Masters (coursework) course at EEE

    • Course run by senior colleague in EEE

    • 2 units of lectures/exams

    • 1 unit of literature research project

    • Students seek supervisors individually

    • Topic chosen by student, but requires approval by supervisor

  • Demographics

    • Almost all are international students; usually East Asian background

    • Technically capable, but generally lack prior experience in conducting research

  • Goal: produce a high quality review paper on chosen topic

  • Mostly negative experiences in 2005

    • Cohort lacked basic literary research skills

    • Unstructured approach towards project

    • Low quality final review papers

Process Details

  • Initial diagnostic task

    • Supervisor supplies two technical papers

      Different levels (magazine, journal) for contrast

    • Students summarise and compare the key points from both sources in one structured, bulleted list

    • Detailed supervisor feedback in written form

  • Knowledge accumulation phase

    • More articles added to the reading list

      Continually add to an organic structured, bulleted list

    • Strategy on further reading

      Student applies critical evaluation on suitability of sources with supervisor input

    • Fortnightly workshops for group presentations & discussions

      Supervisor supplies feedback; optional: external advice (CLPD)

  • Writing phase

    • Student writes review paper based on list

    • Supervisor feedback on first draft approx a week before submission

Outcome and Evaluation

  • Outcomes from 2006

    • Total of 6 students plus1 external (industry) student as control

    • Qualitatively, much improved papers compared to 2005

      • Coherent structures, logical arguments, conciseness, respect of referencing practice

    • Student perspective

      • struggled to cope with demands of research among the pressures of regular coursework

      • English as second language remained a great barrier

  • Framework “matrix” useful for quantitative assessment

  • Unexpected benefit – reduction in plagiarism

    • Turn-it-in software reported major improvements from 2005

    • Reports tend to be on conservative side.

Case 3: Final Year Project (in Progress)

“We have been concerned about a degree of disconnect between the desired leaning outcomes and the assessment methods used in these projects.” HoS of School of EEE, The University of Adelaide.

RSD: Useful First-year to PhD?

Variety of RSD approaches

  • Five distinct approaches have emerged in the use of the RSD so far:

  • Rubrics base to assess the profile of skills for each student, as demonstrated by Human Biology rubrics.  this is the most common approach, and is used by most disciplines.

  • A lock-step approach, whereby students are kept 'in formation' and progressively and corporately develop one level of skills at a time.  This is shown by the Nursing RSD rubrics.

  • Grading with a specific grading, by incorporation into SOLO taxonomy to define grading within a specific level set, as used by Dentistry.

  • Skill Evaluation, to evaluate the skills and levels required by existing assessments.

  • Identification of research skills of higher degree students, Masters by Research and PhD students and/or their supervisors/advisors to locate their present skill set and plot future directions and development needs. 

(SOLO: Structured Observed Leaning Outcomes)

Other Dimensions of RSD

  • Degree of Autonomy

  • Degree of Academic Rigour

  • Degree of Conceptual Demand

  • Degree of disciplinary knowledge required

  • Status of knowledge being pursued

RSD Framework at Program Level !

No studies that consider student outcomes of the explicit development and assessment of research skills over a whole undergraduate or masters-by-coursework program

  • Undergraduate Level

  • Bachelor Oral Health, Adelaide Uni, AU

    • Bachelor of Media Studies , Adelaide Uni, AU

    • Bachelor of Science, Adelaide Uni, AU

    • Bachelor of Nursing, Adelaide Uni, AU

    • Bachelor of Business, Monash Uni, AU

    • Bachelor of Science (Psychology), Macquarie University, AU

    • Postgraduate Level

    • Master of Engineering (Advanced) (Electrical), Adelaide Uni, AU

    • Bridging Program for International PhD students, Adelaide Uni, AU

    • Masters and PhD (Nursing), PhD (Nursing)

    • Trinity College Dublin, Ireland

    • Master of Business, Monash University, AU

RSD in conclusion

  • Provides the Big Picture and relates this to the assessment details for course coordinators, lecturers, tutors, and especially students

  • Informs assessment-first curriculum redesign

  • Same ‘facets’ for multiple assessments, various levels

  • Explicit &Transparent assessment criteria

  • Coherent & Incremental skill development

  • Revisited & (potentially) Cyclic Conceptual Framework

Discussion and Future work

  • How to integrate the framework into other Programs?

  • How will the implementation of RSD framework affect academic workload?

  • How to monitor and assess student’s progress?

  • How can this be implemented for larger class sizes?

  • How can the framework adopted for non-literary research skills?



This session was funded by an Australian Learning and Teaching Council Grant

  • Bennett, N., Dune, E. & Carré, C. (2000). Skills Development in Higher Education and Employment. (Society for Research into Higher Education & Open University Press).

  • Stevens, C. & Fallows, S.J. (2000). Integrating Key Skills in Higher Education: employability, transferable skills and learning for life. Routledge, ISBN 0749432659.

  • Lucas, U., Cox, P., Croudace, C. and Milford, P. (2004). “Who Writes This Stuff?”: Students’ Perceptions of Their Skills Development. Teaching in Higher Education, 9(1), 55-68.

  • Willison, J.W. & O’Regan, K. (2007). Commonly known, commonly not known, totally unknown: A framework for students becoming researchers. Higher Education Research and Development, 26(3).

RSD Web Site

  • Login