Url http www cslu ogi edu sproatr courses aac
Download
1 / 41

URL : cslu.ogi/~sproatr/Courses /AAC/ - PowerPoint PPT Presentation


  • 127 Views
  • Uploaded on

URL : http://www.cslu.ogi.edu/~sproatr/Courses /AAC/. CS506/606: Seminar on Speech and Language Processing for Augmentative and Alternative Communication Spring 2010 Symbol Systems & Even More Symbol Systems. The papers….

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'URL : cslu.ogi/~sproatr/Courses /AAC/' - thais


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Url http www cslu ogi edu sproatr courses aac l.jpg

URL: http://www.cslu.ogi.edu/~sproatr/Courses/AAC/

CS506/606: Seminar on Speech and Language Processing for Augmentative and Alternative CommunicationSpring 2010Symbol Systems & Even More Symbol Systems


The papers l.jpg

SLP for AAC

The papers…

  • Schlosser, R. W. and Sigafoos, J., 2002. Selecting graphic symbols for an initial request lexicon: Integrative review. Augmentative and Alternative Communication. 18(June), 102-123.

  • P. L. Albacete, S. K. Chang and G. Polese. 2006. Iconic language design for people with significant speech and multiple impairments. Assistive Technology and Artificial Intelligence. Lecture Notes in Computer Science, 1458. Springer.

  • Binger, C., Light, J., 2007. The effect of aided AAC modeling on the expression of multi-symbol messages by preschoolers who use AAC. Augmentative and Alternative Communication 23 (1), 30-43.

  • Patel, R., Pilato, S., & Roy, D. (2004). Beyond Linear Syntax: An Image-Orientation Communication Aid. Assistive Technology Outcomes and Benefits, 1(1), 57-67.

  • HamedH. Sad and Franck Poirier. Using Pictographic Representation, Syntactic Information and Gestures in Text Entry. Human-Computer Interaction, Part II, HCII 2009, LNCS 5611, pp. 735–744, 2009.

  • Sutton, A., Soto, G., Blockberger, S., 2002. Grammatical issues in graphical symbol communication. Augmentative and Alternative Communication 18 (September), 192-204.


Schlosser et al l.jpg

SLP for AAC

Schlosser et al.

  • Review of literature on selecting graphic symbols for initial request lexicons

  • Evidence-based practice:

    • Theory based

    • Research based

    • Anecdote based


Issues l.jpg

SLP for AAC

Issues

  • Iconicity & realism

    • How much the symbol resembles its referent

  • Speech output

    • Role of speech output is equivocal

  • Spoken language comprehension of the referent

    • Knowing a word referent seems to assist in learning opaque symbols

  • Concreteness of the referent

    • Role of concreteness of the referent has not been sufficiently studied

  • Reinforcement value of the referent

  • Translucency: “relation between iconicity and reinforcement value of the referent is still largelyunknown.”

  • Correspondence

    • Is there a correspondence between the symbol and the associated action?


Directions for future research l.jpg

SLP for AAC

Directions for Future Research

  • Examine the role of iconicity (opaque versus iconic) on requesting referents of varying reinforcement status (preferred versus nonpreferred).

  • Effects of iconicity (opaque versus iconic) on requesting referents that are receptively known by the learner versus referents that are unknown.

  • Evaluate the role of iconicity in requesting referents that are concrete versus those that are abstract.

  • Investigate whether symbols of varying degrees of iconicity for referents yield differential requesting as a function of the mode of transmission (i.e., speech output versus no speech output).


Implications for clinical practice l.jpg

SLP for AAC

Implications for Clinical Practice


Albacete et al l.jpg

SLP for AAC

Albacete et al

  • “Icon Algebra” + Conceptual Dependency

  • Introduces iconic languages

    • “Finally, there are "natural" iconic languages such as the Chinese ideographs, the Mayan glyphs and the Egyptian pictograms.”

    • This is 牛屎

  • Semantic compaction – Minspeak

    • Map concepts onto multi-meaning icon sentences


Minspeak iconic keyboard l.jpg

SLP for AAC

Minspeak iconic keyboard


Minspeak formalization l.jpg

SLP for AAC

Minspeak formalization

  • APPLE+VERB eat

  • APPLE+NOUN food

  • Each of these icons is polyvalent

  • This paper provides a formalization of the system in terms of iconic algebra

  • Icon is a pair (Xm, Xi), where Xm is the meaning and Xi is the image

    • (This is just Saussure’s notion of signifier and signified)


Iconic operators l.jpg

SLP for AAC

Iconic operators


Iconic operators11 l.jpg

SLP for AAC

Iconic operators

  • MARker:

    • Marks the image of icon Y with the image of icon X to emphasize a local feature. 本

  • CONtextual interpretation

    • Meaning of X considered in the context of Y

  • ENHancement

    • Enhances contextual richness of X by adding attributes of Y

  • INVersion

    • Meaning of X is inverted


Conceptual dependency l.jpg

SLP for AAC

Conceptual dependency

  • Due to Schank

  • Basic principle:

    • For any two sentences that are identical in meaning, regardless of the underlying language, there should be only one representation.

  • Conceptual structure is a network of concepts


Conceptual dependency13 l.jpg

SLP for AAC

Conceptual dependency


Cd for iconic languages l.jpg

SLP for AAC

CD for iconic languages


Inference algorithm l.jpg

SLP for AAC

Inference algorithm

  • Basic module builds a skeleton frame for the iconic sentence by searching for common attribute values in the frames of the single icons in the sentence, according to three match criteria.

  • Second module of the inference algorithm tries to extend the skeleton frame for the iconic sentence by applying iconic operators

  • Third module runs a brute force algorithm to provide further textual descriptions for the iconic sentence, aiming to further enrich its semantic frame.


Example l.jpg

SLP for AAC

Example


Example17 l.jpg

SLP for AAC

Example


Binger light l.jpg

SLP for AAC

Binger & Light

  • Tested aided AAC in effecting multi-symbol messages with 5 preschoolers who used AAC

  • Children who use AAC frequently use one-symbol messages:

    • Lack of communicative efficiency

    • Symbols may encode multiple concepts (ice-cream  “I want an icecream”)

    • Communication partners often ask questions that require only one-word answers

    • There may be other factors due to AAC use

    • Asymmetry between input (speech) and output (AAC) modalities


Aided aac l.jpg

SLP for AAC

Aided AAC

“Essentially, all that is required to convert a spoken model into an aided AAC model is to use natural speech while pointing toand labeling key graphic symbols on the AAC device (e.g., the child points to the symbol MILK, then the adult says More milk while pointing to MOREand MILK).”


Subjects l.jpg

SLP for AAC

Subjects


Digeorge syndrome l.jpg

SLP for AAC

DiGeorge Syndrome

(Wikipedia)

Cognitive impairments

Children with 22q11.2 have a specific profile in neuropsychological tests. They usually have a borderline normal IQ with most individuals having higher scores in the verbal than the nonverbal domains. Cognitive functioning when processing information involving space and time usually shows significant impairment and this generally slows the development of numerical and arithmetical knowledge and skills.

Noteworthy is that these patients are a specifically high-risk group for developing schizophrenia. 30% have at least one incident of psychosis and about a quarter develop actual schizophrenia.


Method l.jpg

SLP for AAC

Method

  • Seven play scenarios

  • Photographs + PCS

  • Independent variable was use of aided AAC


Results l.jpg

SLP for AAC

Results


Results24 l.jpg

SLP for AAC

Results


Results25 l.jpg

SLP for AAC

Results


Prader willi syndrome l.jpg

SLP for AAC

Prader-Willi Syndrome

(Wikipedia)

Individuals with PWS are at risk of learning and attention difficulties. Curfs and Frym (1992) conducted research into the varying degrees of learning disability found in PraderWilli Syndrome (PWS).[12] Their results were as follows:

* 5%: IQ above 85 (average to low average intelligence)

* 27%: IQ 70 – 85 (borderline intellectual functioning)

* 39%: IQ 50 – 70 (mild intellectual disability)

* 27%: IQ 35 – 50 (moderate intellectual disability)

* 1%: IQ 20 – 35 (severe intellectual disability)

* <1%: IQ <20 (profound intellectual disability)


Patel et al l.jpg

SLP for AAC

Patel et al.

  • Proposes an AAC communication aid that is based on “semantic rather than syntactic schema”

  • Relates to the problem of linearization that we discussed in the introductory lecture.

  • Mention a boy named Paul who has problems with his current “linear composition system”


Example element cues howard shane l.jpg

SLP for AAC

Example:Element Cues (Howard Shane)


Teaching language concepts tlc l.jpg

SLP for AAC

Teaching Language Concepts (TLC)


Image oriented messaging interface l.jpg

SLP for AAC

Image-oriented messaging interface


Semantic schema l.jpg

SLP for AAC

Semantic schema


Linearization l.jpg

SLP for AAC

Linearization


Sad poirier l.jpg

SLP for AAC

Sad & Poirier

  • Aimed at text entry on mobile devices:

    • “We propose an interface to enter text that bypasses letter encoding. It’s based on the mental models of meanings already existing in the human mind, and can be processed at very high speed.”

    • Combines graphical elements with gestures


Example34 l.jpg

SLP for AAC

Example


Gestures l.jpg

SLP for AAC

Gestures

  • Down-right: verbs in the future tense

  • Down: verbs in the present tense

  • Down-left: verbs in the past tense

  • Left: delete the last typed word

  • Up-right: adjectives

  • Up: nouns

  • Up-left other parts of speech or grammatical categories


Stats l.jpg

SLP for AAC

Stats

  • Gives direct access to about 1500 common French words

  • 450 words cover about 50% of written/spoken French (they claim)


Key disabling l.jpg

SLP for AAC

Key disabling


Reminds one a bit of science fiction l.jpg

SLP for AAC

Reminds one a bit of science fiction

Since, however, it is inherently impossible in a society based on speech to indicate truly the method of communication of Second Foundationers among themselves, the whole matter will be hereafter ignored. The First Speaker will be represented as speaking in ordinary fashion, and if the translation is not always entirely valid, it is at least the best that can be done under the circumstances.

It will be pretended therefore, that the First Speaker did actually say, "First, I must tell you why you are here," instead of smiling just so and lifting a finger exactly thus.


Sutton et al l.jpg

SLP for AAC

Sutton et al.

  • Examines the effects of graphic symbol communication on syntactic and morphological acquisition


Production issues l.jpg

SLP for AAC

Production issues


Other issues l.jpg

SLP for AAC

Other issues

  • Comprehension issues

  • Acquisition issues

It is hard to

underspecify

in a graphical

representation


ad