Dimensions of Media Object Compehensibility

1. Dimensions of Media Object Compehensibility Lawrie Hunter Kochi University of Technology http://www.core.kochi-tech.ac.jp/hunter/

2. Dimensions of Media Object Compehensibility Island of Shikoku Lawrie Hunter Kochi University of Technology http://www.core.kochi-tech.ac.jp/hunter/ KUT

3. Niigata Osaka Kochi

4. A pattern language for MMC Source of insight: language / language learning Second language (L2) learning: a cognitive process? Comprehension of partially acquired L2: revealing of the nature of text/media. Language learning issues are germane to MMC.

5. A pattern language for MMC Background work towards a human-communication paradigm for the guidance of machines Intervention author's structural model of content information (for second language learning materials) Frame the new multidisciplinary approach of machine-mediated communication Discussion of parameters of difficulty Ground: related issues in second language learning materials Objective development of a pattern language for that multidisciplinary approach to MMC Exemplars ‘considerate text’ ‘considerate multimedia’ Focus factors influencing the difficulty of comprehension of media objects Generating parameters of difficulty in media object comprehension Question how media objects carry information. Tempering: questions of significance and applicability for machine automation.

6. L2 learning materials The creation of second language (L2) learning materials demands document transparency: 1. document purpose 2. document content 3. target behavior 4. target lexical items must all be more immediately apparent to the learner than in the case of materials for L1 medium learning scenarios

7. L2 learning materials The creation of second language (L2) learning materials demands document transparency: 1. document purpose 2. document content 3. target behavior 4. target lexical items work towards transparency is informed by difficulty-related issues difficulty-related issues inform human interaction with info media

8. Earlier work: an EAP tool David Kolb* re using hypertext to present scholarly text: "...the easiest ways of making a complex argument available in HT tend to move the text toward linear structures that do not take full advantage of the possibilities of linked text." *in a talk to KMI at the Open University

9. Earlier work: an EAP tool David Kolb* re using hypertext to present scholarly text: "...the easiest ways of making a complex argument available in HT tend to move the text toward linear structures that do not take full advantage of the possibilities of linked text." "...what the HT can do is present the argument, but also use linkage and juxtaposition to make the reader’s engagement with the argument more creative, self-conscious, and self-critical." *in a talk to KMI at the Open University

10. Earlier work: an EAP tool Lawrie Hunterre using hypertext to present technical L2 text: For the L2 reader, engagement can only be enhanced if the rhetorical and information structures are articulated.

11. Earlier work: an EAP tool Lawrie Hunterre using hypertext to present technical L2 text: For the L2 reader, engagement can only be enhanced if the rhetorical and information structures are articulated. What the HT can do for the NNR/W is tp present simultaneously the various faces of a research paper: the rhetorical moves; the bits of structured information; the text; necessary glosses. * NNR/W EAP = non-native reader/writer of English for Academic Purposes

12. Earlier work: an EAP tool Lawrie Hunterre using hypertext to present technical L2 text: For the L2 reader, engagement can only be enhanced if the rhetorical and information structures are articulated. What the HT can do for the NNR/W is to present simultaneously the various faces of a research paper: the rhetorical moves; the bits of structured information; the text; necessary glosses. And if the NNR/Ws design their personal interface, a negotiated pattern language of NNR/W EAP* will emerge. * NNR/W EAP = non-native reader/writer of English for Academic Purposes

13. Arguably important direction "Tomorrow's literacies... need to be process and systems literacies.” -John Thackara, In the Bubble: Designing in a complex world. MIT Press 2005.

14. Structural view of writing Train of argument Diversions Rhetorical structures This is the domain of structures up here. Extension Functional structures Background Knowledge structures Cohesion devices Rhetorical structure theory, systemic functional linguistics and knowledge structure mapping form a hierarchy of structures, whereas grammar and sentence diagrams reflect rules for texture management. This is the domain of texture down here. Grammar (sentence surface structure)

15. Structural view of writing Train of argument Diversions Rhetorical structures This is the domain of structures up here. Extension Functional structures Background Knowledge structures False hierarchy: the train stops here. Cohesion devices Rhetorical structure theory, systemic functional linguistics and knowledge structure mapping form a hierarchy of structures, whereas grammar and sentence diagrams reflect rules for texture management. This is the domain of texture down here. Grammar (sentence surface structure)

16. L2 reader needs analysis Language skills Argument sequencing Info-structured sentence generation Mimicry of model language Knowledge Niche grammar structures Niche rhetorical structures General register repertoires (distinguishing formal academic from informal academic) Research Paper text structure and information structure Facilities Concordance & collocation resource Bank of model research papers (annotated*) *c.f. Brown and Brown’s ‘annotation’

17. L2 reader wants analysis In a technical hypertext, L2 reader/writers want*: 1. Glossing (of 'difficult' terms and phrases) 2. Moves indicator 3. Lexia position indicator 4. PDF-drawer-like phrase recurrence tab 5. Register converter (e.g. research paper <=> presentation script) 6. Information structure maps for atomic utterances 7. Overall argument map on every lexia (similar to Horn's argument maps or Rhetorical Structure Analysis?) *Based on a survey of 22 PhD engineering students

18. Technical hypertext design: NEEDS A pattern language? www.patternlanguage.com WANTS

19. Technical hypertext design: NEEDS A pattern language? www.patternlanguage.com WANTS • …The language, and the processes which stem from it, merely release the fundamental order which is native to us. They do not teach us, they only remind us of what we know already, and of what we shall discover time and time again, when we give up our ideas and opinions, and do exactly what emerges from ourselves. • Christopher Alexander, • The Timeless Way of Building

20. < $$$ ! Do humans have a GRAPHIC THOUGHT FACILITY? The knowledge structure map is a matrix (confluence) for the situated learner* and the situated mentor to confirm context and the nature of "stolen property."** *Jean Lave **Duguid and Brown

21. < big Hunter’s knowledge structure map links Classification Description Degree comparison Attribute comparison Contrast ! Sequence Cause-effect

22. 2005 project: design level • EEAP* students: HT designs • for the analysis of technical academic papers. • *EEAP = Engineering English for Academic Purposes, • a subset of EAP, • which is a subset of ESP (English for Specific Purposes) Hunter L. (2005) Technical Hypertext Accessibility: Information Structures and Rhetorical Framing. Presentation at HyperText 2005, Salzburg.http://www.lawriehunter.com/presns/%20HT05poster0818.htm

23. TEXT STRUCTURE Introduction Background Question Methods and materials Results Observations Conclusion RHETORICAL MOVES Common knowledge Cite Report Explain Claim Question Qualify Evaluate Decide Infer Project INFO STRUCTURE Describe Classify Compare Sequence Cause-effect Contrast INFOMAP(s) UTTERANCE(s) In general, power plants boil some liquid to make steam, which rotates turbines, which generate electricity. Power plants boil a liquid to produce steam, which is used to rotate turbines, which in turn generate electricity.

24. TEXT STRUCTURE Introduction Background Question Methods and materials Results Observations Conclusion RHETORICAL MOVES Common knowledge Cite Report Explain Claim Question Qualify Evaluate Decide Infer Project INFO STRUCTURE Describe Classify Compare Sequence Cause-effect Contrast INFOMAP(s) UTTERANCE(s) Traditional power plants use fossil fuel heat or heat from nuclear fission to boil water and produce steam at 500°C. Older type power plants boil water with heat from fossil fuel combustion or nuclear fission to produce steam with a temperature of 500°C.

25. TEXT STRUCTURE Introduction Background Question Methods and materials Results Observations Conclusion RHETORICAL MOVES Common knowledge Cite Report Explain Claim Question Qualify Evaluate Decide Infer Project INFO STRUCTURE Describe Classify Compare Sequence Cause-effect Contrast INFOMAP(s) UTTERANCE(s) OTEC power plants use seawater heat to boil ammonia and produce steam at 20°C. OTEC type power plants boil ammonia with the heat of the sea to produce steam with a temperature of 20°C.

26. TEXT STRUCTURE Introduction Background Question Methods and materials Results Observations Conclusion RHETORICAL MOVES Common knowledge Cite Report Explain Claim Question Qualify Evaluate Decide Infer Project INFO STRUCTURE Describe Classify Compare Sequence Cause-effect Contrast INFOMAP(s) UTTERANCE(s) Traditional power plants use fossil fuel heat or heat from nuclear fission to boil water and produce steam at 500°C, whereas OTEC type power plants boil ammonia using the heat of the sea to produce steam with a temperature of 20°C. Older type power plants boil water with heat from fossil fuel combustion or nuclear fission to produce steam with a temperature of 500°C, while OTEC power plants use seawater heat to boil ammonia and produce steam at 20°C.

27. Obstacle in 2005 project Massive diversity in learner perception of knowledge structures.

28. Obstacle in 2005 project Massive diversity in learner perception of knowledge structures.

29. Structural view of writing Train of argument Diversions Rhetorical structures Extension This is the domain of structures up here. Functional structures Background Knowledge structures False hierarchy: the train stops here. Cohesion devices Rhetorical structure theory, systemic functional linguistics and knowledge structure mapping form a hierarchy of structures, whereas grammar and sentence diagrams reflect rules for texture management. This is the domain of texture down here. Grammar (sentence surface structure)

30. Structural view of writing

31. 2006~ new layer: READABILITY The missing link in technical academic writing: Gopen’s readability -subject-verb distance -topic position / stress position -old/new information placement

32. Background: readability work In the design of traditional high-text language learning materials, readability is a prominent concern. Reading difficulty has for some time been seen as depending on -word length -sentence length -text length -number of sentences per paragraph -vocabulary ‘difficulty’ More recent work has extended this list to include -subject-verb distance -adherence to old/new position conventions -topic position/stress position conventions Treated extensively in Hunter L. (1998) Text Nouveau: Visible Structure in Text Presentation. Computer Assisted Language Learning 11(4) pp. 363-379.

33. Background: MM readability Chun, D. M. and Plass, J. L. 1997. Research on text comprehension in multimedia environments. Language learning and technology 1(1): 60-81. Treated extensively in Hunter L. (1998) Text Nouveau: Visible Structure in Text Presentation. Computer Assisted Language Learning 11(4) pp. 363-379.

34. 2006~ new layer: READABILITY Hunter’s new TAW syllabus: assume grammar

35. 2006~ new layer: READABILITY

36. Readability The creation of second language (L2) learning materials demands appropriate readability. 1. understandable by the learner 2. ‘stretching’ learner knowledge/skill 3. contextualized to support stretching 4. orchestrated with degrees of scaffolding

37. Considerate text One inroad to readability is considerate text: Original framing: ･well-written, ･well-organized, and ･signals the organization of its thought to the reader

38. Considerate text One inroad to readability is considerate text: Original framing: ･well-written, ･well-organized, and ･signals the organization of its thought to the reader More recent takes: -glossing -phrase boundary marking -de-idiomatizing -the Plain English movement -graphic organizers -text nouveau

39. Text nouveau is still text Text comprehension in multimedia environments is a rich variant, BUT : Chun, D. M. and Plass, J. L. 1997. Research on text comprehension in multimedia environments. Language learning and technology 1(1): 60-81.

40. Text comprehension in multimedia environments is a rich variant, BUT : Text nouveau is still text Chun, D. M. and Plass, J. L. 1997. Research on text comprehension in multimedia environments. Language learning and technology 1(1): 60-81.

41. Sharing considerate text L2 learning objects: welcome to the TagTower of Babel! KUT English is a Moodle department. Appropriateness of learning materials/tasks is very complex. Tagging of these materials & tasks is daunting.

42. Sharing considerate text Appropriateness of learning materials/tasks is very complex. Tagging of these materials & tasks is daunting. Fortunately, as David Weinberger points out*, there is a huge amount of metadata out there, but this allows multiple simultaneous organizations of content. *June 12, 2007 interview with IT Conversations http://www.itconversations.com/shows/detail1838.html Weinberger books The Cluetrain Manifesto Small Pieces Loosely Joined Everything is Miscellaneous

43. Considerate multimedia? Considerate text in the context of M3C suggests the notion of considerate multimedia Tentative definition: considerate multimodal objects are those which contain few non-essential obstacles to their comprehension.

44. Considerate multimedia? Considerate text in the context of M3C suggests the notion of considerate multimedia Tentative definition: considerate multimodal objects are those which contain few non-essential obstacles to their comprehension. Tentative definition 2: considerate multimodal objects are those which are tagged for various forms of comprehension difficulty.

45. Considerate multimedia?

46. Multimedia comprehensibility? “Considerate multimedia” confronts vastly more complexity than considerate text Alternative approach: create a set of parameters for multimedia comprehensibility One approach to comprehensibility: explore obstacles to comprehensibility, as has been done in readability work.

47. Multimodal equivalent of readability In the domain of multimodal computer-mediated communication, the question of readability translates as ease of comprehension: How easy is it for a human to extract all the information contained in a multimodal media object (MMO)?

48. Parameters of media object function To measure the ease of extraction of all the INTENDED information contained in a MMO, we need a characterization of the difficulty of extraction: Tentative list of sources of information extraction difficulty, for simplicity’s sake limited here to text objects graphic objects speech objects video objects and combinations thereof. This is a tentative, exploratory framing of MMO comprehensibility,

49. Learning object tag ‘shopping list’ Parameters of media object function *utterance: minimal spoken, written or graphical communication unit

50. Concept density Concept density* in text space Concept density* in aural time Concept density* in video space *concept density = number of exophoric references per sentence/minute/frame Parameters of media object function