Designing Multimodal Discourses
This presentation is the property of its rightful owner.
Sponsored Links
1 / 27

Designing Multimodal Discourses PowerPoint PPT Presentation


  • 82 Views
  • Uploaded on
  • Presentation posted in: General

Designing Multimodal Discourses. Bai Hong’ai. College of Foreign Languages, YBU. monomodality : Language as the central and only full means for representation and communication . means of expression (material and other technologies). multimodality.

Download Presentation

Designing Multimodal Discourses

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Designing multimodal discourses

Designing Multimodal Discourses

Bai Hong’ai

College of Foreign Languages, YBU


Designing multimodal discourses

monomodality :

Language as the central and only full

means for representation and communication


Designing multimodal discourses

means of expression (material and other technologies)

multimodality

When humans converse with one another, we utilize a wide array of mediato interact, including spoken language, gestures, and drawings.

We exploit multiple human sensory systems or modes of communication, including vision, audition, and taction.

the manner of expression


Designing multimodal discourses

Some media and modes of communication are more efficient or effective than others for certain tasks, users, or contexts

e.g. the use of speech ---

control devices in hand and eyes-

busy contexts,

the use of maps ---

convey terrain and

cartographic

information


Designing multimodal discourses

Communication is defined as a process in which a semiotic product or event is both articulated or produced and interpreted or used, and a semiotic product or event may involve multiple modes and media.

The design process of this multimodality semiotic product or event, is to be explored in the present paper, i.e. the generation of multimodal discourses


Designing multimodal discourses

I. What is multimodal discourse

Multimodal discourse is defined as“the use of several semiotic modes in the design of a semiotic product or event, together with the particular way in which these modes are combined”– they may for instance reinforce each other (‘say the same thing in different ways’), fulfil complementary roles, or be hierarchically ordered, as in action films, where action is dominant, with music adding a touch of emotive colour and sync sound a touch of realistic ‘presence’ (Kress G. & Van Leeuwen, T, 1996, 2001; Batesman et al., 2002)


Designing multimodal discourses

This suggests that we need to expand our conception of mode to embrace multimodal discourse analysis (hereafter MDA).

MDA draws upon recent work in the field of social semiotics in an attempt to theorise how discourses are generated in a variety of modes through a variety of media, and to investigate empirically the design decisions employed and how these have been distributed across the presentation modes available.


Designing multimodal discourses

II. How to design a multimodal discourse

STEP 1: The Process of Designing a multimodal discourse

Content planning – determining what information should be presented;

Technique selection – selecting appropriate media or modalities which will be used for realization, and coordinating the selections for specific information content;

Presentation design – determining how media or modalities will be used to communicate their selected information content;

Coordination – composing, organizing, resolving conflicts and maintaining presentation consistency.


Designing multimodal discourses

Figure 1: Feedback and Multiple Knowledge Sources involved in the Processes

Presentation Design

Content Planning

Technique Selection

Coordination

Data Task Context Discourse

Characteristics Characteristics State


Designing multimodal discourses

STEP 2: Determine what and how media or modes to be used

Myers (1999: 91)

Different media convey different information

Different media go with different practices

Spoken and written words can limit, link and extend images

Words (including typography) define voice and audience


Designing multimodal discourses

STEP 3: Establishing the right modality: Naturalistic modality, Abstract modality and Sensory modality

Naturalistic modality: produced with sound, images, audio-visual texts or the combination of all these.

Abstract modality: with regard to images

Sensory modality: based on the sensory pleasure or displeasure represented by an audio-visual text


Designing multimodal discourses

‘larger-than-life’ or “more-than-real” images or sounds


Designing multimodal discourses

III. Advertisements:

typical multimodal discourses


Designing multimodal discourses

Medium with sound only--- Radio

MAN: I… I… I love you.

WOMAN: Awwww.

MAN: No, I mean … I … I mean. I don’t … just …

WOMAN: What (GIGGLES), go ahead …

MAN: I just … don’t think I’d be anything without you …

WOMAN: Aw … that’s so nice …

MAN: I, uh, made these for you …

(Baking pan and foil sounds.)

WOMAN: Aw … you didn’t have to do this …

MAN: They’re just brownies.

WOMAN: Ooh, and they’re still warm too.


Designing multimodal discourses

MAN: I was gonna get you something good.

WOMAN: Aww, don’t be silly, could I have one?

MAN: (Laughs.) Well, yeah sure … go ahead …

WOMAN: (Mouth full.) You know … ?

MAN: What … you don’t like brownies?

WOMAN: No … no … (Slight gag.) miggkkk …

MAN: I’m sorry. I can’t understand you.

WOMAN: Neet … dmiggkk.

MAN: You need Mick?

WOMAN: Miccliggkk!

MAN: A guy named Mick! I can’t believe this …

WOMAN: Pleeeease miiiggggggllkk


Designing multimodal discourses

MAN: Yeah, yeah, okay you don’t have to scream his name in ecstasy. I

get the message. All right I got it, I’m gone.

WOMAN: No, umbhum, no, miiiiiiggglllkk!

MAN: Look, I gotta go, I’m sorry I’m gonna need my CDs back … and

WOMAN: No, no, miiiggllk.

MAN: … and I’m sorry for this whole thing …

WOMAN: … No, miiigggllkk …

MAN: No, it’s Scott remember ?

WOMAN: No, miigggkkkffff …

ANNOUNCER: True love means never having to say, “Got Milk?”


Designing multimodal discourses

Medium with image, sound, colour, words, etc.

--- internet banner ads, TV ads


Designing multimodal discourses

V. Implications to Language teaching

There is a need to re-conceptualise the term ‘literacy’ as a multimodal process. As language is beginning to be de-centred and image and other modes are increasingly foregrounded, the traditional conceptions of learning and ‘literacy’ as entirely language based are inadequate.


Designing multimodal discourses

Multimodal discourse with language de-centred


Designing multimodal discourses

Literacy, as a complex process, is multimodally mediated, in which the modes of speech and writing are only ever one part of the multimodal modes, just the same as image, colour, sound and animated movement on screen.


Designing multimodal discourses

We need to pay attention to the substance of learning including new learning materials. In recent years, educationalists have focused upon learning environments as social and personal settings and forms of interaction. It is time to take note of the fact that learning is about something and that “something” is communicated through a range of media – from books and photocopies, through audiovisual media to computer media and in the near future perhaps also mobile media. Only with the advent of a range of new media it is perhaps made clear to us that learning and media were always intimately increasingly intertwined.


Designing multimodal discourses

Thank you for your attention!


  • Login