Multimodal reformulation during shared
Sponsored Links
This presentation is the property of its rightful owner.
1 / 24

Overview PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Multimodal reformulation during shared synchronous note-taking : potential pedagogical consequences for teachers and students Kristine Lund*, Gregory Dyke°, Jean-Jacques Girardot° *ICAR Research Lab, CNRS, University of Lyon °RIM Department, Ecole des Mines de St. Etienne. Overview.

Download Presentation


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Multimodal reformulation during shared synchronous note-taking : potential pedagogical consequences for teachers and studentsKristine Lund*, Gregory Dyke°, Jean-Jacques Girardot°*ICAR Research Lab, CNRS, University of Lyon°RIM Department, Ecole des Mines de St. Etienne


  • Motivation

    • Note-taking and multimodal reformulation

  • The corpus

    • A series of tool-supported meetings

      • tutoring computer programming projects at l’Ecole des Mines de St. Etienne

  • Tatiana (Trace Analysis Tool for Interaction ANAlysts)

    • Synchronization of multi-source data, analyses and visualizations

  • Results

    • Form and content / pedagogical consequences ?

  • Synthesis and perspectives


  • Listen, understand and assimilate, choose what to record, study (Suritsky & Hughes, 1991)

    • Modify definition for our context…(e.g. also reading, but no trace of studying)

  • Look for patterns in note-taking and determine efficiency in relation to students’ results

    • The value of detailed notes rises with time between the time notes are taken and the test (Williams & Eggert, 2002) ;

    • A strong correlation between the quantity of notes taken and exam performance (Nye, et. al. 1984).

  • Note-taking as a tool for transforming knowledge and not as data collection (Castello & Monereo, 2005)

    • We focus on the process of note-taking

  • Modes

    • Abstract resources “non-material” that help us to make sense

      • touch, hearing, smell, taste (senses),

      • gesture, speech (means of expression)

  • Media

    • Specific material forms in which modes are expressed

      • The human body, a mechanical body, computer (physical apparatus)

  • Example of a mode

    • Manipulate an object to write

  • Example of four media

Multimodal interactions (and reformulations)

Kress & Van Leeuwen (2001) ; Bellik & Teil (1992)


  • Collaborative writing / note-taking context

  • An oralo-graphic situation, articulates two different semantic modes

    • Apotheloz (2001)Cooperation in this task consists in “showing in a continuous way, each step of the way, the manner in which what is formulated articulates with what has already been formulated”

A multimodal corpus (talk, writing)

  • First year computer programming projects at Ecole des Mines in St. Etienne

    • 33 tutoring sessions, lasting from 30 to 60 minutes a piece (PPF --> 12 transcribed)

    • 9 groups

    • 3 project types, each chosen by three groups.

    • These 9 groups were tutored by 2 professors.

      • Computer traces of note-taking and transcribed dialogue

      • The computer program / final product

      • Final reports

      • The presentations

      • Grades

Exploratory research questions

  • What are the nature of the multimodal reformulations ?

  • How can they demonstrate conceptual transformations ?


Tatiana - Trace Analysis Tool for Interaction ANAlysts

  • Synchronize multi-source data

    • Computer traces of human interaction

    • Videos and their corresponding transcription (dialogue, etc.)

  • Replay in order to understand activity

  • Define analysis units and categorize facets of the interaction

  • Visualize the categorizations on a temporal line

  • Regroup, re-categorize, re-visualize

  • Share

Transcription of talk


Synchronize the multi-source data, replay and think about units of analysis / Research Q

Shared text editor where notes are taken (replayed)

Log file

Create the note-taking writing units (done manually)

Log file

Note-taking units

Categorize the transcription of talk

Categorize the reformulated - reformulation pair (done manually)

Categorize the note-taking units

Categorization schema

Categorized transcription of talk

Create the links between the reformulated - reformulation pair (manual)

Categorized note-taking units

Directional link between the reformulated and the reformulation

Transcription of talk

Visualization (automatically created)

A’s note-taking

  • Synchronize multi-source data

  • Replay in order to understand activity

  • Define analysis units and categorize facets of the interaction

  • Visualize the categorizations on a temporal line

  • Regroup, re-categorize, re-visualize

  • Share

B’s note-taking

The reformulated

The reformulation

Tool replayer (5)

Log Data (1)

Tatiana - example analysis of corpus

Writing units (2)

Video (6)

Transcription (3)


Writing units

Visualization (4)


control (7)


Highlights are synchronized

Results - session 1

Results - session 2

A writing conflict provokes a spatial organization

Results - session 2

Chronological time-lag between monologue and note-taking

Results - session 2

Differences in the structure of reformulation

Results - session 2

Reformulation of tutor dialogue written and co-elaborated by two students

Results - session 2

Tutor reaction upon seeing note-taking

Synthesis, interpretations and questions

  • Multimodal reformulation, co-elaborative written reformulation of dialogue

    • Not yet seen oral reformulation of co-elaborative note-taking

  • From written to oral

    • Response to a student question;

    • Incite students to take note-taking further (conceptual deepening)

      • Is this still reformulation ?

  • From oral to written

    • Anticipation of teacher talk ; do such students understand content better ?

    • How to understand the choices students make when reformulating ?

    • What is the linguistic nature of these reformulations ?

    • How do the characteristics of the shared text editor influence note-taking and co-elaboration? (technical affordances of resources)

      • Collaborative note-taking ?

    • For what type of content is there a time-lag between teacher talk and note-taking ?


  • More closely study the nature of these multimodal reformulations

  • How can they illustrate knowledge transformation ?

    • We don’t look to measure student performance in terms of exam results as a result of note-taking.

    • Rather, we want to quality the transformation of conceptual knowledge through characterizing reformulation.

    • What are the pedagogical consequences ?


  • How can the shared text editor characteristics influence note-taking ?

    • More or less co-elaboration ?

  • Can note-taking be understood as a sort of formative (vs. summative) evaluation ?

  • What about auto and hetero-regulation ? And vicarious learning?

Thanks to the PPF !

  • Transcription of dialogue from videos

  • Communication between TATIANA and CARTE

    • Collection, activity Analysis and Regulation based on Traces Enriched (Christophe Courtin, Stéphane Talbot / Chambery)

      • to facilitate automating certain analysis tasks (e.g. definition of note-taking units, categorization)

      • to highlight relevant sequences of events in large numbers of TATIANA traces

    • CARTE supplies the following software tools :

      • convert TATIANA traces CARTE traces and vice versa

      • import/export CARTE traces to and from a corpus of traces

      • Analyze CARTE traces by means of rules (pre-defined patterns)

You can download Tatiana


    [email protected]

    [email protected]

    [email protected]

    [email protected]

  • Login