Barbara Louwagie – Piet Desmet
This presentation is the property of its rightful owner.
Sponsored Links
1 / 36

Barbara Louwagie – Piet Desmet PowerPoint PPT Presentation


  • 98 Views
  • Uploaded on
  • Presentation posted in: General

Barbara Louwagie – Piet Desmet. An empirical study on the influence of feedback on learning performance for second language learners of French using a web-based CALL environment: E xperimental design & results of the preliminary study. Objectives.

Download Presentation

Barbara Louwagie – Piet Desmet

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Barbara louwagie piet desmet

Barbara Louwagie – Piet Desmet

An empirical study on the influence of feedback on learning performance for second language learners of French using a web-based CALL environment:Experimental design & results of the preliminary study

Eurocall 2006 – Granada


Objectives

Objectives

  • The development of (automatic) feedback is a central point in CALL. =>The general purpose of our study: feedback = effective supplementary opportunity for learning?

  • Aim of the preliminary study: optimalization of the design

  • => This presentation is about the search for an adequate design

Eurocall 2006 – Granada


Structure

Structure

  • Definition of the object of study

  • Preliminary study

  • New design

Eurocall 2006 – Granada


Definition of feedback

Definition of feedback

“In the context of teaching in general, feedback is information that is given to the learnerabout his or her performance of a learning task, usually with the objective of improving this performance.”

(Ur 2003)

Eurocall 2006 – Granada


Feedback

Feedback?

feedback

evaluation

quantitative

qualitative

(corrective)

summative

formative

Eurocall 2006 – Granada


Barbara louwagie piet desmet

Corrective feedback in formative evaluation

  • Aim: to create learning opportunities

  • Consists of

    • identification of errors committed

    • explanation of errors (cognitive strategies)

    • suggestions to correct the errors

      (Janssens ea 2003)

Eurocall 2006 – Granada


Advantages fb in call

Advantages FB in CALL

  • immediate (Heift 2001)

  • individualized (Kornum 1989)

  • objective (Kornum 1989)

Eurocall 2006 – Granada


Barbara louwagie piet desmet

  • Verify if the feedback provided in the learning environment used at our university is effective

  • Definition of ‘efficacy’ or ‘effectiveness’“The power to produce effects or intended results. This suggests a strong causal relationship between an intervention, such as the use of a particular item of technology in a learning situation and a discernible change in the learning process, the learning climate or the learning achievement.”

    (Felix 2005)

Eurocall 2006 – Granada


Preliminary study structure

Preliminary study: structure

  • Aim

  • Design

  • Subjects

  • Procedure

  • Learning Environment (Idioma-tic)

  • Statistical analyses

  • Results

  • Evaluation of the preliminary study

Eurocall 2006 – Granada


Preliminary study aim

Preliminary study: aim

  • To verify the effect of different kinds of feedback on the score obtained for the final test

  • To test the design of the study

Eurocall 2006 – Granada


Design

Design

  • Feedback configurations

  • Components & themes

  • Other variables

Eurocall 2006 – Granada


5 feedback configurations schematical representation

5 feedback configurations, schematical representation

no feedback

feedback

closed activity

closed activity

half-open activity

general & specific FB

intermediary FB

general, specific & error-specific FB

intermediary, general & specific FB

Eurocall 2006 – Granada


Barbara louwagie piet desmet

  • 5 themes, 3 components:

    • Speech acts

      • Telephone conversation

    • Vocabulary

      • Business vocabulary

      • Quantitative vocabulary

    • Grammar

      • Conjunctions

      • Grammatical orthography

  • Different feedback conditions for each of these themes

Eurocall 2006 – Granada


Type of fb as within subjects factor

Type of FB as ‘within-subjects’ factor

Eurocall 2006 – Granada


Other variables

+ other variables

  • Personal variables(sex, level of proficiency in French, learning style (Kolb-test), educational background, …)

  • Situational variables(subject of the exercises, number of tries, time spent on a certain exercise, …)

Eurocall 2006 – Granada


Subjects

Subjects

  • n = 107

  • Undergraduate students in applied economic science & commercial engineering

  • Learners of French as a second language

  • 5 groups (n ~ 20)

  • Stratification solely based on current branch of study, otherwise at random

Eurocall 2006 – Granada


Procedure

Procedure

Duration: 9 weeks

Week 1

  • (short demo of Idioma-tic)

  • Orientation testitemset 1, 50 MC-questions

    Week 3

  • Practice session 1 (1 ½ hour)itemset A1, A2, A3, A4, A5 (± 30 questions)

    Week 5

  • Practice session 2 (1 ½ hour) itemset B1, B2, B3, B4, B5 (± 30 questions)

    Week 7

  • Practice session 3 (1 ½ hour) itemset C1, C2, C3, C4, C5 (± 30 questions)

    Week 9

  • Evaluation testitemset 2 (5*(3+3) items)

Eurocall 2006 – Granada


Learning environment

Learning environment

  • Authoring tool IDIOMA-TIC

  • Tracking and logging system

Eurocall 2006 – Granada


Tracking logging

Tracking & logging


Overview of activities 2000

Overview of activities (± 2000)


Statistical analyses

Statistical analyses*

  • Cross-tabulations (tabulate procedure)

  • Correlations (corr procedure)

  • Multi-level analysis (REML-algorithm**)

    * Many thanks to M. Vandewaetere

    ** residual restricted maximum likelihood

Eurocall 2006 – Granada


Results descriptive statistics

Results: Descriptive statistics

Cross-tabulation (tabulate procedure)Distribution for sex, high school curriculum and current curriculum

Eurocall 2006 – Granada


Barbara louwagie piet desmet

  • Distribution for exercise series and current curriculum

Eurocall 2006 – Granada


Analytic statistics

Analytic statistics

Correlations (corr procedure)

  • Indication of influence of FB, but just below statistical significance

  • Some obvious significant results:

    • Correlation between the number of exercises made and the final score ( = 0.13, α = 0.0036)

Eurocall 2006 – Granada


Multi level analysis i

Multi-level analysis I

Multiple themes per subject are measured (5). Themes are clustered for any single student.

  • Feedback has a significant effect (F(4,415) = 2.99, p < 0.05 )

  • Theme has a significant effect (F(4,415) = 69.84, p < 0.0001)

Eurocall 2006 – Granada


Additional remarks

Additional remarks

  • The foregoing analysis is not complete

  • There are repeated measuring moments (3) for any single student.The scores obtained during these measuring moments for a single subject will correlate more strongly than the scores of different subjects on a single measuring moment.

  • The following analysis takes into account the repeated mesures design.

Eurocall 2006 – Granada


Multi level analysis ii

Multi-level analysis II

Multiple themes per subject are measured (5); moreover, each theme is measured three times. Themes are clustered for any single moment, and the moments are clustered for any single student.

  • There is a difference between the three measuring moments (F(2,202) = 68.55, p < 0.0001)

  • There is a difference between the types of feedback (F(4,424) = 1456.89, p < 0.0001)

  • There is a difference between the themes(F(4,424) = 114.39, p < 0.0001)

Eurocall 2006 – Granada


Barbara louwagie piet desmet

  • These significant main effects (previous slide) can, however, not be interpreted unequivocally,

    • because the two-way interaction terms are significant

      • feedback*moment, F(8,768) = 15.52, p < 0.0001

      • theme*moment, F(8,720) = 9.17, p < 0.0001

    • Id. for three-way interaction

      • thema*feedback*moment: F(48,720) = 8.97, p < 0.0001

  • Nevertheless, first indication of influence of FB

  • => Optimize the design to obtain significant results

Eurocall 2006 – Granada


Evaluation of the preliminary study

Evaluation of the preliminary study

  • Some practical problems

    • Over-parametrization (1000 cells per subject) => many interaction effects

    • Only one observation per cell: 2 themes can not be compared

    • Stratification solely on the basis of current curriculum

  • Personal variables

    • Maintain these variables

    • Level of proficiency in French: Orientation test: OK

    • Learning style test – Kolb test: not validated psychometrically speaking, the results can not be validated

Eurocall 2006 – Granada


Barbara louwagie piet desmet

  • Exercise series

    • Re-usable

    • Pay attention to differences in the level of difficulty between themes (grammatical orthography is comparatively easier than the four other themes)

  • Authoring tool:

    • Stable learning environment

    • Tracking & logging: no scoring provided for half-open assignments

Eurocall 2006 – Granada


New design

New design

  • Add qualitative study (questionnaires)

  • Feedback configurations: maintain 5 types

  • Themes: simplification: 5 => 3 themes

    • Speech acts: telephone conversation

    • Vocabulary: business vocabulary

    • Grammar: conjunctions

  • Other variables: maintain and extend:

    • Self-evaluation as to knowledge of and attitude towards French (motivation)

    • Familiarity with the on-line tool

Eurocall 2006 – Granada


Type of fb as a between subject factor

Type of FB as a ‘between-subject’ factor

Eurocall 2006 – Granada


Procedure1

Procedure

  • Orientation testitemset 1, MC questions

    • Exercise moment 1itemset A1, A2, A3, A4, A5

  • evaluation 1itemset 2 (20 closed assignments, 20 half-open assignments)

    • Exercise moment 2itemset B1, B2, B3, B4, B5

  • evaluation 2itemset 3 (different from 2)

    • Exercise moment 3itemset C1, C2, C3, C4, C5

  • evaluation 3itemset 4 (different from 2 & 3)

Eurocall 2006 – Granada


Barbara louwagie piet desmet

  • Extended stratification:

    • current curriculum,

    • sex,

    • score for orientation test,

    • educational background, …

  • Each time trial run

  • Items at random, for 2 reasons:

    • Concentration effect

    • Time lapse

Eurocall 2006 – Granada


More information

More information

Presentation: http://www.kuleuven-kortrijk.be/ALT/

Piet Desmet

Professor of French and Applied Linguistics

K.U.Leuven (Campus Kortrijk)

[email protected]

www.ling.arts.kuleuven.ac.be/franling_e/pdesmet

Barbara Louwagie

K.U.Leuven Campus Kortrijk

[email protected]

Eurocall 2006 – Granada


  • Login