Explorative Thread-based Analysis of Patterns of Collaborative Interaction in Chat

1. Explorative Thread-based Analysis of Patterns of Collaborative Interaction in Chat Nan Zhou Murat Cakir

2. Overview • Motivation for thread based sequential analysis • Information about the data and the coding scheme • A simple mathematical model for performing thread based sequential analysis of chat • 4 research questions • Extra: A possible method based on our model to support identification of the type of interaction (Exploratory vs Expository)

3. Motivation • Analysis of fine-grained patterns of interaction is important for understanding collaborative learning • Content based analysis cannot reveal the sequential unfolding of interaction • A sequential analysis of postings is necessary to study how actions of participants unfold through interaction • A naïve sequential analysis based on observed ordering of postings could be misleading due to artificial turn orderings produced by the quasi-synchronous chat medium • By considering threads we aim to address this issue. • The idea is to take into account the more complex linking structures while conducting sequential analysis.

4. Data • 6 coded transcripts • In 3 cases the problem was announced in advance • In the latter cases the problem was announced at the beginning of the session • We will consider Powwow2a, 2b and 18 in this presentation • We will focus on 4 dimensions: Conversation, Conversation Thread, Problem Solving, and Problem Solving Thread

5. The Coding Scheme • Math collaboration dimensions scheme_version 15_8-6-04.doc

6. What is captured by the Conversation Thread? • The conversation thread aims… • To tie postings of a single member together which span multiple lines • Setup, elaboration and extension codes are used for this purpose • To tie two postings where the latter one conversationally replies to the first one. Each statement is posted by different users. • E.g. a question-explain pair, an offer-agree pair • Such pairs can be considered as an approximation of adjacency pairs in chat • The shape of the graph induced by the c-thread is a tree

7. Threaded view (based on C-thread)

8. What is captured by the Problem Solving Thread? • PB-Thread links a posting to an earlier one from a problem solving perspective. • Some parts are inherited from the conversation thread: • Explain, Elaboration, Critique, Agree, Disagree, Follow • Restatements are linked to the original statement, a summary is linked to the statements that are summarized • having multiple links are possible in these cases, which is not the case in the conversation thread • The graph induced by the problem solving thread need not be a tree

9. The computational model I • Input: Coded transcript • 2 graph representations based on c-thread and pb-thread dimensions are generated • Each node of a graph corresponds to an object capturing properties of each posting (e.g. its author, statement, codes etc.)

10. The computational model II • Traversals are performed over graphs to identify frequently occurring sequential patterns. • We consider two postings Pi-Pj as a sequential pattern if they are linked by the thread • First, we analyzed dyads and triads to study local organization of interaction • Dyads and triads are the most frequently occurring patterns in a group of 3-4 participants • We also consider the unfolding of local structures in the whole discussion

11. The computational model III A variable Ci can be replaced by 1. the author name, 2. the conversation code, 3. the problem solving code, 4. a combination of conversation and problem solving codes. Dyad pattern Triad patterns

12. Research Questions • Research Question 1: What patterns of interaction are frequently observed in a synchronous, collaborative math problem solving environment? • Research Question 2: How can patterns of interaction be used to identify: (a) each member’s level of participation; (b) the distribution of contributions among participants; and, (c) whether participants are organized into subgroups through the discussion? • Research Question 3: What are the most frequent patterns related to the main activities of the math problem solving? How do these patterns sequentially relate to each other? • Research Question 4: What are the (most frequent) minimal building blocks observed during “local” interaction? How are these local structures sequentially related together yielding larger interactional structures?

13. RQ1: Interactional Patterns a percentage matrix for dyads based on conversation codes

14. RQ1: Interactional Patterns a row-based percentage matrix for the conversation dyads

15. RQ1: frequent conversational patterns • Source vs. Sink • Request-Response: 16%, 7%, 9%, 9%, 10%, 8% • Response-Response: 12%, 5%, 2%, 4%, 10%, 11% • State-Response: 8%, 6%, 4%, 2%, 5%, 16% • Setup-X :8%, 14%, 12%, 2%, 3%, 4% • X-Extension: 14%, 15%, 9%, 7%, 9%, 6%

16. RQ1 cont’d • Implication of Setup-X and X-Extension • Pruning: combine the fragmented statements into a single node

17. RQ2: identify participation -- dyads

18. RQ2: identify participation -- triads • Percentage of triads having the same author: 15% for group A vs. 42% for group B • Elaboration • Anti-symmetry patterns: • MCP to REA 23% vs. REA to MCP 14% • AVR-PIN: 17%, 18%; AVR-SUP: 13%, 13%

19. RQ2: identify participation -- triads (cont’d) • Who initiated the triads? • the percentage of triads initiated by each member: AVR PIN SUP OFF 41% 29% 20% 7%

20. RQ3: Problem Solving Patterns • Problem solving activities are defined by clustering problem solving codes. • Orientation, Tactic, Strategy: • Signaled action: understanding the problem statement and/or proposing strategies to approach the problem • Perform, Result: • Signaled action: proposed strategies are being executed • Restate, Summary: • Signaled action: Help a member to catch up and/or producing a reformulation of the problem • Check, Reflect: • These are interposed among the 3 activities described above (not considered as an individual cluster)

21. RQ3: Problem Solving Patterns

22. RQ4 Maximal Patterns • Previous slide showed the difference between the types of problem solving activities performed by each group • Maximal structure (the sequential unfolding of problem solving activities) can be used to contrast both groups at a global scale

23. RQ4 Maximal Patterns • Powwow2a • 1.Orientation activity in which the group identified a relevant sub-problem to work on • 2. The group executed the strategy to find the altitudes and areas of each triangle. • 3. Reflective activity in which they tried to relate the solution of the sub-problem to the general problem. They realized they made a mistake in the formula • 4. Session ended without a solution • Powwow2b • 1. Each member solved the problem individually, no orientation activity. • 2. Each member revealed their solution steps (captured as perform and result codes). • 3. When they started working on the extra credit portion a short orientation activity was performed • 4. They individually executed their strategies and reported their steps. • 5. Session ended with a correct solution (both to the original and to the extra credit portion), which was not co-constructed

24. Extra: Using thread-analysis for studying the type of interaction. • Alan and Fatos presented two different types of interaction in powwows (expository versus exploratory interaction) based on their CA analysis of powwow2a and powwow18 • We will try to present additional evidence supporting this observation • For this purpose we will contrast powwow2a and powwow18 in terms of their sequential organizations

25. Observations on powwow2a • As concluded by Alan & Fatos before, interaction is organized in an ‘explorative’ way • Suggesting tactics, co-constructing ideas for approaching the problem • Interesting pattern: High percentage of Strategy, Orientation, Tactic codes

26. Observations on powwow18 • There are 2 episodes in which one member shares his/her solution with other members. • Solution steps are revealed step-by-step • Other members’ reaction to each step with Agree, Follow statements • suggesting the presenter to continue with the next step • Interesting pattern: ‘Offer’s followed by ‘Follow’ and ‘Agree’s, high percentage of execution actions (i.e. perform, result) • As concluded by Alan and Fatos, interaction is organized in an ‘expository’ way

27. Contrasting problem solving activities Less exploration work is done in powwow18, high percentage of execution steps

28. Contrasting local organization of conversation 270 O 271 F Offer – Follow – Follow 272 F 273 El 274 F Elaboration – Follow – Follow 275 F 276 F These are called triads

29. Triads that fit to the pattern we observed in powwow18 Powwow2a Powwow18

30. Triads linking 3 distinct authors, which is another property of the patterns we observed in powwow18 Group 2 Group 1

31. Contrasting powwow2a vs powwow18: Summary • More effort is devoted to explorative activities in powwow2a as opposed to powwow18 • More effort is devoted to exposing solution steps in powwow18, which is an indication of expository interaction. • Powwow18 has a higher percentage of triad patterns that fit to expository interaction • This short analysis shows that thread analysis could be useful in studying the type of the interaction

32. Conclusion • We presented an alternative way for doing sequential analysis of interaction in chat, based on threads. • We presented methods for making assessments about • Each participant’s level of participation • The conversational structure of discussion • Problem solving activities performed by the group • This is an ongoing work: • We will study more factors (e.g. expository vs exploratory) • We will build a statistical model