Natural language processing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

Natural Language Processing PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

Natural Language Processing. Lecture Notes 1. Today. Administration and Syllabus course web page Introduction. Natural Language Processing. What is it? What goes into getting computers to perform useful and interesting tasks involving human languages.

Download Presentation

Natural Language Processing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

Natural language processing

Natural Language Processing

Lecture Notes 1



  • Administration and Syllabus

    • course web page

  • Introduction

Natural language processing1

Natural Language Processing

  • What is it?

    • What goes into getting computers to perform useful and interesting tasks involving human languages.

    • Secondarily: insights that such computational work give us into human languages and human processing of language.

Natural language processing2

Natural Language Processing

  • Foundations are in computer science (AI, theory, algorithms,…); linguistics; mathematics; logic and statistics; and psychology

Why should you care

Why Should You Care?

  • Two trends

    • An enormous amount of knowledge is now available in machine readable form as natural language text

    • Conversational agents are becoming an important form of human-computer communication

Knowledge of language

Knowledge of Language

  • Words (words and their composition)

  • Syntax (structure of sentences)

  • Semantics (explicit meaning of sentence)

  • Discourse and pragmatics (implicit and contextual meaning)

Small applications

Small Applications

  • Line breakers

  • Hyphenators

  • Spelling correctors

  • Optical Character Recognition software

  • Grammar and style checkers

Big applications

Big Applications

  • Question answering

  • Conversational agents

  • Text summarization

  • Machine translation

Natural language processing


NLP, as in many areas of AI:

  • We’re often dealing with ill-defined problems

  • We don’t often come up with perfect solutions/algorithms

  • We can’t let either of those facts get in our way

Course material

Course Material

  • We’ll be intermingling discussions of:

    • Linguistic topics

      • Syntax and meaning representations

    • Computational techniques

      • Context-free grammars

    • Applications

      • Translation and QA systems

Chapter 1

Chapter 1

  • Knowledge of language

  • Ambiguity

  • Models and algorithms

  • History

Knowledge of language1

Knowledge of Language

  • Phonetics and phonology: speech sounds, their production, and the rule systems that govern their use

  • Morphology: words and their composition from more basic units

    • Cat, cats (inflectional morphology)

    • Child, children

    • Friend, friendly (derivational morphology)

Knowledge of language2

Knowledge of Language

  • Syntax: the structuring of words into legal larger phrases and sentences



  • The meaning of words and phrases

    • Lexical semantics: the study of the meanings of words

    • Compositional semantics: how to combine word meanings

    • Word-sense disambiguation

      • River bank vs. financial bank



  • Indirect speech acts:

    • Do you have a stapler?

  • Presupposition:

    • Have you stopped beating your wife?

  • Deixis and point of view:

    • Zoe was angry at Joe. Where was he?

  • Implicature:

    -Yes, there are 3 flights to Boston. In fact, there are 4.

    * The general was assassinated. In fact, he isn’t dead.



  • Utterance interpretation in the context of the text or dialog

    • Sue took the trip to New York. She had a great time there.

      • Sue/she;

      • New York/there;

      • took/had (time)



  • Almost all of the non-trivial tasks performed by NLP systems are ambiguity resolution tasks

  • There is ambiguity at all levels of language



  • I saw the woman with the telescope

  • Syntactically ambiguous:

    • I saw (NP the woman with the telescope)

    • I saw (NP the woman) (PP with the telescope)

I made her duck

“I made her duck”

  • I cooked waterfowl for her

  • I cooked waterfowl belonging to her

  • I create the duck she owns

  • I caused her to lower her head quickly…

  • Part of speech tagging: is “duck” a noun or verb?

  • Parsing syntactic structure: is “her” part of the “duck” NP?

  • Word-sense disambiguation (lexical semantics): does “make” mean create, lower head, or cook?

Dealing with ambiguity

Dealing with Ambiguity

  • Two approaches:

    • Tightly coupled interaction among processing levels; knowledge from other levels can help decide among choices at ambiguous levels.

    • Pipeline processing

  • Most NLP systems are probabilistic: they make the most likely choices

Models and algorithms

Models and Algorithms

  • Models (as we are using the term here):

    • Formalisms to represent linguistic knowledge

  • Algorithms:

    • Used to manipulate the representations and produce the desired behavior

      • choosing among possibilities and combining pieces



  • State Machines: finite state automata, finite state transducers

  • Formal rule systems: context free grammars

  • Logical formalisms: first-order predicate calculus; higher-order logics

  • Models of uncertainty: Bayesian probability theory

  • Vector Space Models



  • Many of the algorithms that we’ll study will turn out to be transducers; algorithms that take one kind of structure as input and output another.



  • In particular..

    • State-space search

      • To manage the problem of making choices during processing when we lack the information needed to make the right choice

    • Dynamic programming

      • To avoid having to redo work during the course of a state-space search

    • Machine Learning (classifiers, EM, etc)

State space search

State Space Search

  • States represent pairings of partially processed inputs with partially constructed answers

    • E.g. sentence + partial parse tree

  • Goal is to arrive at the right/best structure after having processed all the input.

    • E.g. the best parse tree spanning the sentence

  • As with most interesting AI problems the spaces are too large and the criteria for “bestness” are difficult to encode (so heuristics, probabilities)

Dynamic programming

Dynamic Programming

  • Don’t do the same work over and over.

  • Avoid this by building and making use of solutions to sub-problems that must be invariant across all parts of the space.

  • Login