Natural language processing
1 / 26

Natural Language Processing - PowerPoint PPT Presentation

  • Uploaded on

Natural Language Processing. Lecture Notes 1. Today. Administration and Syllabus course web page Introduction. Natural Language Processing. What is it? What goes into getting computers to perform useful and interesting tasks involving human languages.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about ' Natural Language Processing' - ginger

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript


  • Administration and Syllabus

    • course web page

  • Introduction

Natural language processing1
Natural Language Processing

  • What is it?

    • What goes into getting computers to perform useful and interesting tasks involving human languages.

    • Secondarily: insights that such computational work give us into human languages and human processing of language.

Natural language processing2
Natural Language Processing

  • Foundations are in computer science (AI, theory, algorithms,…); linguistics; mathematics; logic and statistics; and psychology

Why should you care
Why Should You Care?

  • Two trends

    • An enormous amount of knowledge is now available in machine readable form as natural language text

    • Conversational agents are becoming an important form of human-computer communication

Knowledge of language
Knowledge of Language

  • Words (words and their composition)

  • Syntax (structure of sentences)

  • Semantics (explicit meaning of sentence)

  • Discourse and pragmatics (implicit and contextual meaning)

Small applications
Small Applications

  • Line breakers

  • Hyphenators

  • Spelling correctors

  • Optical Character Recognition software

  • Grammar and style checkers

Big applications
Big Applications

  • Question answering

  • Conversational agents

  • Text summarization

  • Machine translation


NLP, as in many areas of AI:

  • We’re often dealing with ill-defined problems

  • We don’t often come up with perfect solutions/algorithms

  • We can’t let either of those facts get in our way

Course material
Course Material

  • We’ll be intermingling discussions of:

    • Linguistic topics

      • Syntax and meaning representations

    • Computational techniques

      • Context-free grammars

    • Applications

      • Translation and QA systems

Chapter 1
Chapter 1

  • Knowledge of language

  • Ambiguity

  • Models and algorithms

  • History

Knowledge of language1
Knowledge of Language

  • Phonetics and phonology: speech sounds, their production, and the rule systems that govern their use

  • Morphology: words and their composition from more basic units

    • Cat, cats (inflectional morphology)

    • Child, children

    • Friend, friendly (derivational morphology)

Knowledge of language2
Knowledge of Language

  • Syntax: the structuring of words into legal larger phrases and sentences


  • The meaning of words and phrases

    • Lexical semantics: the study of the meanings of words

    • Compositional semantics: how to combine word meanings

    • Word-sense disambiguation

      • River bank vs. financial bank


  • Indirect speech acts:

    • Do you have a stapler?

  • Presupposition:

    • Have you stopped beating your wife?

  • Deixis and point of view:

    • Zoe was angry at Joe. Where was he?

  • Implicature:

    -Yes, there are 3 flights to Boston. In fact, there are 4.

    * The general was assassinated. In fact, he isn’t dead.


  • Utterance interpretation in the context of the text or dialog

    • Sue took the trip to New York. She had a great time there.

      • Sue/she;

      • New York/there;

      • took/had (time)


  • Almost all of the non-trivial tasks performed by NLP systems are ambiguity resolution tasks

  • There is ambiguity at all levels of language


  • I saw the woman with the telescope

  • Syntactically ambiguous:

    • I saw (NP the woman with the telescope)

    • I saw (NP the woman) (PP with the telescope)

I made her duck
“I made her duck”

  • I cooked waterfowl for her

  • I cooked waterfowl belonging to her

  • I create the duck she owns

  • I caused her to lower her head quickly…

  • Part of speech tagging: is “duck” a noun or verb?

  • Parsing syntactic structure: is “her” part of the “duck” NP?

  • Word-sense disambiguation (lexical semantics): does “make” mean create, lower head, or cook?

Dealing with ambiguity
Dealing with Ambiguity

  • Two approaches:

    • Tightly coupled interaction among processing levels; knowledge from other levels can help decide among choices at ambiguous levels.

    • Pipeline processing

  • Most NLP systems are probabilistic: they make the most likely choices

Models and algorithms
Models and Algorithms

  • Models (as we are using the term here):

    • Formalisms to represent linguistic knowledge

  • Algorithms:

    • Used to manipulate the representations and produce the desired behavior

      • choosing among possibilities and combining pieces


  • State Machines: finite state automata, finite state transducers

  • Formal rule systems: context free grammars

  • Logical formalisms: first-order predicate calculus; higher-order logics

  • Models of uncertainty: Bayesian probability theory

  • Vector Space Models


  • Many of the algorithms that we’ll study will turn out to be transducers; algorithms that take one kind of structure as input and output another.


  • In particular..

    • State-space search

      • To manage the problem of making choices during processing when we lack the information needed to make the right choice

    • Dynamic programming

      • To avoid having to redo work during the course of a state-space search

    • Machine Learning (classifiers, EM, etc)

State space search
State Space Search

  • States represent pairings of partially processed inputs with partially constructed answers

    • E.g. sentence + partial parse tree

  • Goal is to arrive at the right/best structure after having processed all the input.

    • E.g. the best parse tree spanning the sentence

  • As with most interesting AI problems the spaces are too large and the criteria for “bestness” are difficult to encode (so heuristics, probabilities)

Dynamic programming
Dynamic Programming

  • Don’t do the same work over and over.

  • Avoid this by building and making use of solutions to sub-problems that must be invariant across all parts of the space.