Measuring the impact of a pedagogical change or innovation
This presentation is the property of its rightful owner.
Sponsored Links
1 / 33

Measuring the impact of a pedagogical change or innovation PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

Measuring the impact of a pedagogical change or innovation. Strategies for Dissemination . Anastassis Kozanitis Educational consultant Polytechnique Montréal. Do you know anyone in this room who has innovated ?. 1- no 2- yes 3- I’m not sure.

Download Presentation

Measuring the impact of a pedagogical change or innovation

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Measuring the impact of a pedagogical change or innovation

Measuring the impact of a pedagogical change or innovation

Strategies for Dissemination

Anastassis Kozanitis

Educational consultant

Polytechnique Montréal


Do you know anyone in this room who has innovated

Do youknowanyone in thisroomwho has innovated?

  • 1- no

  • 2- yes

  • 3- I’m not sure


Have you innovated in the past 2 years

Haveyouinnovatedin thepast 2 years?

  • 1- no

  • 2- yes

  • 3- I’mnotsure

  • 4- I would of liked to, but I lacktheressources


Why did you not innovate or why do you don t want to innovate

Whydidyou (not) innovateorwhy do you (don’t) want to innovate?

  • Activity 1: Think-pair-share


Supporting elements institutional or departmental levels

Supporting elements? Institutional or departmental levels

  • Pedagogical culture

  • Pedagogical training for teachers

  • Educational support center

  • Valuing teaching (support of the institution or department)

  • Tangible recognitions (awards, $, career progress)

  • Educational leadership of the authorities (Dean, Head of department, etc.)

  • Support (human, material and financial resources)

  • Work collaboratively within a program approach


Elements that inhibit or push back

Elements that inhibit or push back

  • Doubt that it will work.

  • Negative reaction from students.

  • Sensing that there was no added value.

  • Material, financial or contextual constraints (e.g. group size), lack of time or work load too large.

  • Unsuccessful blind trial and error.


Supporting elements individual level

Supporting elements? Individual level

  • Strength of your personal beliefs

  • Desire to make your work more interesting

  • Professionalization of teaching (3 levels of intensity)

    • 1) Reflective practice (Schön, 1983)

    • 2) Scholarly teaching (Richlin, 2001)

    • 3) Scholarship of teaching and learning (Potter & Kustra, 2011)


Outline and goals

Outline and goals

  • Choose what elements to measure as a result of a pedagogical innovation implementation

  • Identify how to measure and analyze its impact on students

  • Develop strategies to disseminate the findings and motivate colleagues to do the same.

  • Commit to practice a scholarly teaching


Circle of innovation

Circle of innovation


Pedagogical innovation definition

pedagogical innovationDefinition:

  • A deliberate activity that aims to introduce novelty in a given context, and seeks to substantially improve the learning of students in a situation of interaction or interactivity (Béchard & Pelletier, 2002)


Initiatives for innovations with or without ict

Initiatives for innovationswith or without ICT

  • Institutional

    • Changes in software platform(Moodle)

    • Curricular changes (competency-based curriculum)

  • Departmental

    • Curricular changes(program-approach)

    • Structural changes (course fusion)

  • Individual

    • Changes to active learning methods


The active learning continuum

The Active learning continuum*

Prolonged time or large scale activities

Limited time or small scale activities

Interactive lectures

Student centered

Instructor centered

Think-pair-share

Case study

Project-basedlearning

Roleplaying

*Adaptedfrom Michael Prince (2011). Active cooperativelearning. BucknellUniversity. http://www.asee.org/documents/conferences/annual/2011/plenary-michael-prince.pdf


Teaching methods

Teachingmethods

Instructor centered

Student centered

  • Problem-based learning

  • Guided discovery

  • Case study

  • Project-based learning

  • Role playing

  • Simulation

  • Tournament

  • Peer instruction

  • Team-based learning

  • Laboratory

  • Discussions

  • Lecturing

  • Podcasting

  • Video-conferencing

  • Demonstration


Is there something you wish to change in your teaching

is there something you wish to change in your teaching?

  • Whatproblemshaveyouobservedconcerningyourstudents’ learning?

  • Whatsolutionshaveyoutried and whatweretheresults?

  • Example: implementinga new method

    • Problem-basedlearning

    • Project-basedlearning

    • Case studies

    • Team-basedlearning

    • Other?


Relevant pre active questions

relevant Pre-active Questions

  • How can I fix my mistakes from last semester?

  • How can I get my students’ attention?

  • What learning activities should I provide to help students understand the subject?

  • How can I arouse their interest and motivation?

  • What content-specific resources do I need?

  • How will I consider student heterogeneity and their different ways of learning?

  • What are my expectations about student productions?


Activity 2 current situation desired situation

Activity 2 current situation – Desired situation

  • Step 1.

  • Identify, in your practice, a specific situation you want to improve or a recurring problem you wish to solve.

  • Step 2.

  • Using one or two words, post your situation or problem on a piece of paper for everyone to see.

  • For example:

    • Classroom participation

    • Motivation

    • Critical thinking

    • Student attitude


Scholarship of teaching and learning sotl

Scholarship of teaching and learning (SoTL)

  • Definition

  • A careful analysis from the teacher about students’ learning, before and/or after a systematic and intentional change of pedagogy, with an evaluation of the results of this change, and dissemination of these results (Potter y Kustra, 2011).


Steps to sotl

Steps to Sotl

  • 1 – write a research question 2 - design the study 3 - collect data 4 - analyze the data 5 - present and publish the results


Specific aspects to be measured

Specific aspects to be measured

  • For example, if a teacher says he has problems with teaching large groups, you may look at the following indicators :

    • Absenteeism rate

    • Verbal or non-verbal behavior of students in the classroom

    • Lack of student participation during activities

    • Grades on tests and assignments

    • Results of teaching evaluations

    • Student comments


Activity 3 choosing the specific aspects to measure or assess

Activity 3choosing the specific aspects to measure or assess

  • Identify the specific aspects of the general theme you want to measure.

  • You have to be very accurate. For example, it is not enough to say I want to measure motivation. But rather what aspect of motivation (interest in the course or learning activity, perception of usefulness of what they are learning, amount of time or effort invested in the course, the perception of self-efficacy, etc..).

  • You can measure multiple aspects, however make sure that the information you seek is:

    available, valid, accurate, and true.


Activity 4 identify how to measure or assess the specific aspects

Activity 4identify how to measure or assess the specific aspects

  • It is the procedural or operational part

  • What instrument or process do you need in order to measure the specific aspects

  • 2 options:1) You can choose and adapt an existing instrument 2) You can build or create an instrument

  • In both cases you must validate the instrument

    • How will you do the validation?

    • Who are your sources?

    • With how many participants are going to validate the instrument?


Methods or tools for measuring the impact of the pedagogical change

methods or tools for measuring the impact of the pedagogical change

  • Quantitative methods

  • Quasi-experimental design

  • Likert type questionnaire

  • Existing tools like the NSSE

  • Comparing means (ANOVA, t-test)

  • Regressions

  • SEM o Path analysis

  • Qualitative methods

  • Case study

  • Semi-structured interviews (NVivo)

  • Focus group

  • Quality circle

  • In situ observations using a form


Examples of methods

Examples of methods

  • A valid questionnaire on motivation;

  • Some open-ended questions regarding opinions;

  • A test or an examination of knowledge or skills;

  • An observation form of classroom behavior;

  • Self-assessments, peer-assessments, student evaluations;

  • Etc.


Dissemination

dissemination

  • How to get your colleagues involved?

  • How to convince them to change?

  • How can you teach them what you have learned?

  • What are the usual arguments for inaction?


Dissemination1

dissemination

cognitive

affective


Factors related with pedagogical innovation development b chard pelletier 2002

Factors related with pedagogical innovation development (Béchard & Pelletier, 2002)

  • 4 categories :

    • External environment

    • Institution

    • Department

    • Classroom


Estrategias

Estrategias


Opportunities and threats

opportunities and threats


Strategies for dissemination

Strategies for dissemination

  • Organizing an event for presentation of best practices (institutional or department level)

  • Formal and informal pedagogical training

  • Mentoring – coaching

  • Teamwork with a colleague giving the same course

  • Participating in educational conferences (STLHE, AERA, CHEP, Lilly conference, ISSOTL, etc.)

  • Funding pedagogical initiatives (specific criteria)


What can lead to innovations

What can lead to innovations?

1 - Personal, informal initiatives 2 - Evaluation of teaching 3 – Work in collegialityto prepare assessment activities 4 - Work in collegially to prepare course material 5 - Promotion of educational initiatives (funds, special events e.g. Teaching day) 6 - Scholarship of teaching and learning (SoTL) 7 - Training and pedagogical support for teachers 8 - Paradigm shift (from teaching to learning) 9 - International collaboration projects


Activity 7

Activity 7

  • Share with a neighbor what other strategies are adequate given your context.

  • Think of any resistance you anticipate.


Conclusion

conclusion

  • Degree of satisfaction with the workaccomplished?

  • Degree of satisfaction with the help?

  • Any questions remain?

Obrigado!


References

References

  • Béchar, J-P. & Pelletier, P. (2002). Dynamique des innovations pédagogiques en enseignement supérieur: à la recherche d’un cadre théorique. Available online at:http://neumann.hec.ca/oipg/fichiers/2002-002_-_Dynamique_des_innovations_pedagogiques_en_enseignement_superieur.pdf

  • Cornet, J. & Voz, G. (2010). «Comment former de futurs étudiants réflexif?» Education and formation, [En ligne], octobre, e-294, http://ute3.umh.ac.be/revues/include/download.php?idRevue=10&idRes=81.

  • Hannan, A., English, S. & Silver, H. (1999). Whyinnovate? Somepreliminaryfindingsfrom a researchprojecton “Innovations in teaching and learning in highereducation”. Studies in HigherEducation, 24(3), 279-289.

  • Kapur, M. (2012). Productive failure. Available online at: http://www.isls.org/icls2012/downloads/K2Kapur.pdf

  • Lenoir, Y. (2012). Analyse réflexive : un outil de questionnement sur la gestion des activités d’enseignement-apprentissage, Outil 2. Vivre le primaire, 25(4).

  • O’Brien, M. (2008). Navigating the SoTLLandscape: A Compass, Map and Some Tools for GettingStarted. International Journal for the Scholarship of Teaching and Learning, 2(2), 1-20.

  • Perrenoud, P. (2001). Développer la pratique réflexive dans le métier d'enseignant: professionnalisation et raison pédagogique: ESF éditeur.

  • Potter, M. K., & Kustra, E. (2011). The relationshipbetweenscholarlyteaching and SoTL: Models, distinctions, and clarifications. International Journal for the Scholarship of Teaching and Learning.

  • Schön, D.A. (1983). The Reflective Practitioner. San Francisco: Jossey-Bass.


  • Login