Measuring the impact of a pedagogical change or innovation
1 / 33

Measuring the impact of a pedagogical change or innovation - PowerPoint PPT Presentation

  • Uploaded on

Measuring the impact of a pedagogical change or innovation. Strategies for Dissemination . Anastassis Kozanitis Educational consultant Polytechnique Montréal. Do you know anyone in this room who has innovated ?. 1- no 2- yes 3- I’m not sure.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Measuring the impact of a pedagogical change or innovation' - xylia

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Measuring the impact of a pedagogical change or innovation

Measuring the impact of a pedagogical change or innovation

Strategies for Dissemination

Anastassis Kozanitis

Educational consultant

Polytechnique Montréal

Do you know anyone in this room who has innovated
Do youknowanyone in thisroomwho has innovated?

  • 1- no

  • 2- yes

  • 3- I’m not sure

Have you innovated in the past 2 years
Haveyouinnovatedin thepast 2 years?

  • 1- no

  • 2- yes

  • 3- I’mnotsure

  • 4- I would of liked to, but I lacktheressources

Why did you not innovate or why do you don t want to innovate
Whydidyou (not) innovateorwhy do you (don’t) want to innovate?

  • Activity 1: Think-pair-share

Supporting elements institutional or departmental levels
Supporting elements? Institutional or departmental levels

  • Pedagogical culture

  • Pedagogical training for teachers

  • Educational support center

  • Valuing teaching (support of the institution or department)

  • Tangible recognitions (awards, $, career progress)

  • Educational leadership of the authorities (Dean, Head of department, etc.)

  • Support (human, material and financial resources)

  • Work collaboratively within a program approach

Elements that inhibit or push back
Elements that inhibit or push back

  • Doubt that it will work.

  • Negative reaction from students.

  • Sensing that there was no added value.

  • Material, financial or contextual constraints (e.g. group size), lack of time or work load too large.

  • Unsuccessful blind trial and error.

Supporting elements individual level
Supporting elements? Individual level

  • Strength of your personal beliefs

  • Desire to make your work more interesting

  • Professionalization of teaching (3 levels of intensity)

    • 1) Reflective practice (Schön, 1983)

    • 2) Scholarly teaching (Richlin, 2001)

    • 3) Scholarship of teaching and learning (Potter & Kustra, 2011)

Outline and goals
Outline and goals

  • Choose what elements to measure as a result of a pedagogical innovation implementation

  • Identify how to measure and analyze its impact on students

  • Develop strategies to disseminate the findings and motivate colleagues to do the same.

  • Commit to practice a scholarly teaching

Circle of innovation
Circle of innovation

Pedagogical innovation definition
pedagogical innovationDefinition:

  • A deliberate activity that aims to introduce novelty in a given context, and seeks to substantially improve the learning of students in a situation of interaction or interactivity (Béchard & Pelletier, 2002)

Initiatives for innovations with or without ict
Initiatives for innovationswith or without ICT

  • Institutional

    • Changes in software platform(Moodle)

    • Curricular changes (competency-based curriculum)

  • Departmental

    • Curricular changes(program-approach)

    • Structural changes (course fusion)

  • Individual

    • Changes to active learning methods

The active learning continuum
The Active learning continuum*

Prolonged time or large scale activities

Limited time or small scale activities

Interactive lectures

Student centered

Instructor centered


Case study



*Adaptedfrom Michael Prince (2011). Active cooperativelearning. BucknellUniversity.

Teaching methods

Instructor centered

Student centered

  • Problem-based learning

  • Guided discovery

  • Case study

  • Project-based learning

  • Role playing

  • Simulation

  • Tournament

  • Peer instruction

  • Team-based learning

  • Laboratory

  • Discussions

  • Lecturing

  • Podcasting

  • Video-conferencing

  • Demonstration

Is there something you wish to change in your teaching
is there something you wish to change in your teaching?

  • Whatproblemshaveyouobservedconcerningyourstudents’ learning?

  • Whatsolutionshaveyoutried and whatweretheresults?

  • Example: implementinga new method

    • Problem-basedlearning

    • Project-basedlearning

    • Case studies

    • Team-basedlearning

    • Other?

Relevant pre active questions
relevant Pre-active Questions

  • How can I fix my mistakes from last semester?

  • How can I get my students’ attention?

  • What learning activities should I provide to help students understand the subject?

  • How can I arouse their interest and motivation?

  • What content-specific resources do I need?

  • How will I consider student heterogeneity and their different ways of learning?

  • What are my expectations about student productions?

Activity 2 current situation desired situation
Activity 2 current situation – Desired situation

  • Step 1.

  • Identify, in your practice, a specific situation you want to improve or a recurring problem you wish to solve.

  • Step 2.

  • Using one or two words, post your situation or problem on a piece of paper for everyone to see.

  • For example:

    • Classroom participation

    • Motivation

    • Critical thinking

    • Student attitude

Scholarship of teaching and learning sotl
Scholarship of teaching and learning (SoTL)

  • Definition

  • A careful analysis from the teacher about students’ learning, before and/or after a systematic and intentional change of pedagogy, with an evaluation of the results of this change, and dissemination of these results (Potter y Kustra, 2011).

Steps to sotl
Steps to Sotl

  • 1 – write a research question 2 - design the study 3 - collect data 4 - analyze the data 5 - present and publish the results

Specific aspects to be measured
Specific aspects to be measured

  • For example, if a teacher says he has problems with teaching large groups, you may look at the following indicators :

    • Absenteeism rate

    • Verbal or non-verbal behavior of students in the classroom

    • Lack of student participation during activities

    • Grades on tests and assignments

    • Results of teaching evaluations

    • Student comments

Activity 3 choosing the specific aspects to measure or assess
Activity 3choosing the specific aspects to measure or assess

  • Identify the specific aspects of the general theme you want to measure.

  • You have to be very accurate. For example, it is not enough to say I want to measure motivation. But rather what aspect of motivation (interest in the course or learning activity, perception of usefulness of what they are learning, amount of time or effort invested in the course, the perception of self-efficacy, etc..).

  • You can measure multiple aspects, however make sure that the information you seek is:

    available, valid, accurate, and true.

Activity 4 identify how to measure or assess the specific aspects
Activity 4identify how to measure or assess the specific aspects

  • It is the procedural or operational part

  • What instrument or process do you need in order to measure the specific aspects

  • 2 options:1) You can choose and adapt an existing instrument 2) You can build or create an instrument

  • In both cases you must validate the instrument

    • How will you do the validation?

    • Who are your sources?

    • With how many participants are going to validate the instrument?

Methods or tools for measuring the impact of the pedagogical change
methods or tools for measuring the impact of the pedagogical change

  • Quantitative methods

  • Quasi-experimental design

  • Likert type questionnaire

  • Existing tools like the NSSE

  • Comparing means (ANOVA, t-test)

  • Regressions

  • SEM o Path analysis

  • Qualitative methods

  • Case study

  • Semi-structured interviews (NVivo)

  • Focus group

  • Quality circle

  • In situ observations using a form

Examples of methods
Examples of methods

  • A valid questionnaire on motivation;

  • Some open-ended questions regarding opinions;

  • A test or an examination of knowledge or skills;

  • An observation form of classroom behavior;

  • Self-assessments, peer-assessments, student evaluations;

  • Etc.


  • How to get your colleagues involved?

  • How to convince them to change?

  • How can you teach them what you have learned?

  • What are the usual arguments for inaction?




Factors related with pedagogical innovation development b chard pelletier 2002
Factors related with pedagogical innovation development (Béchard & Pelletier, 2002)

  • 4 categories :

    • External environment

    • Institution

    • Department

    • Classroom

Strategies for dissemination
Strategies for dissemination

  • Organizing an event for presentation of best practices (institutional or department level)

  • Formal and informal pedagogical training

  • Mentoring – coaching

  • Teamwork with a colleague giving the same course

  • Participating in educational conferences (STLHE, AERA, CHEP, Lilly conference, ISSOTL, etc.)

  • Funding pedagogical initiatives (specific criteria)

What can lead to innovations
What can lead to innovations?

1 - Personal, informal initiatives 2 - Evaluation of teaching 3 – Work in collegialityto prepare assessment activities 4 - Work in collegially to prepare course material 5 - Promotion of educational initiatives (funds, special events e.g. Teaching day) 6 - Scholarship of teaching and learning (SoTL) 7 - Training and pedagogical support for teachers 8 - Paradigm shift (from teaching to learning) 9 - International collaboration projects

Activity 7
Activity 7

  • Share with a neighbor what other strategies are adequate given your context.

  • Think of any resistance you anticipate.


  • Degree of satisfaction with the workaccomplished?

  • Degree of satisfaction with the help?

  • Any questions remain?



  • Béchar, J-P. & Pelletier, P. (2002). Dynamique des innovations pédagogiques en enseignement supérieur: à la recherche d’un cadre théorique. Available online at:

  • Cornet, J. & Voz, G. (2010). «Comment former de futurs étudiants réflexif?» Education and formation, [En ligne], octobre, e-294,

  • Hannan, A., English, S. & Silver, H. (1999). Whyinnovate? Somepreliminaryfindingsfrom a researchprojecton “Innovations in teaching and learning in highereducation”. Studies in HigherEducation, 24(3), 279-289.

  • Kapur, M. (2012). Productive failure. Available online at:

  • Lenoir, Y. (2012). Analyse réflexive : un outil de questionnement sur la gestion des activités d’enseignement-apprentissage, Outil 2. Vivre le primaire, 25(4).

  • O’Brien, M. (2008). Navigating the SoTLLandscape: A Compass, Map and Some Tools for GettingStarted. International Journal for the Scholarship of Teaching and Learning, 2(2), 1-20.

  • Perrenoud, P. (2001). Développer la pratique réflexive dans le métier d'enseignant: professionnalisation et raison pédagogique: ESF éditeur.

  • Potter, M. K., & Kustra, E. (2011). The relationshipbetweenscholarlyteaching and SoTL: Models, distinctions, and clarifications. International Journal for the Scholarship of Teaching and Learning.

  • Schön, D.A. (1983). The Reflective Practitioner. San Francisco: Jossey-Bass.