Natural language processing lecture 1 introduction to nlp
This presentation is the property of its rightful owner.
Sponsored Links
1 / 52

Natural Language Processing Lecture 1 : Introduction to NLP PowerPoint PPT Presentation


  • 194 Views
  • Uploaded on
  • Presentation posted in: General

Natural Language Processing Lecture 1 : Introduction to NLP. Winter 2014-15 Lecturer: Prof. Roi Reichart. Course Information. Lecture: Roi Reichart. TA: Ira Leviant Class: Thursday 11:30 - 14:30, TA: Thursday 10:30 - 11:30. Course Prerequisites.

Download Presentation

Natural Language Processing Lecture 1 : Introduction to NLP

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Natural language processing lecture 1 introduction to nlp

Natural Language ProcessingLecture 1: Introduction to NLP

Winter 2014-15

Lecturer: Prof. Roi Reichart


Course information

Course Information

  • Lecture: Roi Reichart. TA: Ira Leviant

  • Class: Thursday 11:30 - 14:30, TA: Thursday 10:30 - 11:30


Course prerequisites

Course Prerequisites

  • Basic Linear Algebra, Probability, Algorithms

  • Machine learning will help (but some of it will be reviewed in class and TA sessions)

  • Programming skills


Course grading

Course Grading

  • Class and TA attendance – 5% (mandatory, contact me if you cannot attend)

  • Student lecture – 20% (last hour every week)

  • 40% - four home work assignments (one programming question plus a few theoretical ones)

  • 35% - final project (?)


Talk outline

Talk Outline

  • Language and Language Technology

  • Language as a test case for scientific modeling

  • What is NLP ?

  • Basic NLP tasks (focus of this course)

  • NLP Applications

  • Why is NLP hard (ambiguity)


Natural language processing nlp

Natural Language Processing (NLP)


Natural language processing nlp1

Natural Language Processing (NLP)


Natural language processing nlp interdisciplinary

Natural Language Processing (NLP) - Interdisciplinary


Natural language processing nlp technology

Natural Language Processing (NLP) - Technology


Talk outline1

Talk Outline

  • Language and Language Technology

  • Language as a test case for scientific modeling

  • What is NLP ?

  • Basic NLP tasks (focus of this course)

  • NLP Applications

  • Why is NLP hard (ambiguity)


Modeling language learning and processing

Modeling Language learning and Processing

The four components of scientific description of a phenomenon

  • Theory

  • Model

  • Parameter estimation, a.k.a learning

  • Prediction, a.k.a inference


Modeling language learning and processing1

Modeling Language Learning and Processing

The four components of scientific description of a phenomenon

  • Theory – many layers of language analysis are hidden (i.e. does not exist in the world)

  • Model

  • Parameter estimation, a.k.a learning

  • Prediction, a.k.a inference


Modeling language learning and processing2

Modeling Language Learning and Processing


Modeling language learning and processing3

Modeling Language Learning and Processing

The four components of scientific description of a phenomenon

  • Theory – many layers of language analysis are hidden (i.e. does not exist in the world)

  • Model – since many of the layers are hidden there can be strong disagreement on their modeling

  • Parameter estimation, a.k.a learning

  • Prediction, a.k.a inference


Modeling language learning and processing4

Modeling Language Learning and Processing

The four components of scientific description of a phenomenon

  • Theory – many layers of language analysis are hidden (i.e. does not exist in the world)

  • Model – since many of the layers are hidden there can be strong disagreement on their modeling

  • Parameter estimation, a.k.a learning – as models contain hidden layers this can be complicated

  • Prediction, a.k.a inference – as models contain hidden layers this can be complicated


Talk outline2

Talk Outline

  • Language and Language Technology

  • Language as a test case for scientific modeling

  • What is NLP ?

  • Basic NLP tasks (focus of this course)

  • NLP Applications

  • Why is NLP hard (ambiguity)


What is in nlp

What is in NLP ?

  • Machine Learning and Optimization (especially structured prediction)


What is in nlp1

What is in NLP ?

  • Machine Learning and optimization (especially structured prediction)

  • Statistical Modeling (parameter estimation, inference)


What is in nlp2

What is in NLP ?

  • Machine Learning and optimization (especially structured prediction)

  • Statistical Modeling (parameter estimation, inference)

  • Linguistics


What is in nlp3

What is in NLP ?

  • Machine Learning and optimization (especially structured prediction)

  • Statistical Modeling (parameter estimation, inference)

  • Linguistics

  • Cognitive Science


Talk outline3

Talk Outline

  • Language and Language Technology

  • Language as a test case for scientific modeling

  • What is NLP ?

  • Basic NLP tasks (focus of this course)

  • NLP Applications

  • Why is NLP hard (ambiguity)


Language processing

Language Processing


Basic nlp tasks

Basic NLP Tasks

  • Basic input analysis (word segmentation):


Basic nlp tasks1

Basic NLP Tasks

  • Basic input analysis (word segmentation):


Basic nlp tasks2

Basic NLP Tasks

  • Word level analysis (morphological segmentation):


Basic nlp tasks3

Basic NLP Tasks

  • Sentence level analysis: Syntax, Part-of-Speech (POS) tagging:

I gave the smaller balls to Itamar

PRP V DT JJ N IN N

PRP – Personal Pronoun JJ - Adjective

V – Verb N – Noun

DT – Determiner IN - Preposition


Basic nlp tasks4

Basic NLP Tasks

  • Sentence level analysis: Syntax, syntactic parsing:

I gave the ball to Itamar

PRP VBD DT NN TO NNP


Basic nlp tasks5

Basic NLP Tasks

  • Sentence level analysis: lexical semantics:


Basic nlp tasks6

Basic NLP Tasks

  • Word meaning (semantics):


Basic nlp tasks7

Basic NLP Tasks

  • Multilingual Processing:


Talk outline4

Talk Outline

  • Language and Language Technology

  • Language as a test case for scientific modeling

  • What is NLP ?

  • Basic NLP tasks (focus of this course)

  • NLP Applications

  • Why is NLP hard (ambiguity)


Text processing

Text Processing

A powerful car bomb exploded today in Baghdad inside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington.

The blast came only two days after another car bomb exploded in a crowdedstreet in Mosul in the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Together with the shooting in Najaf three weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq …


Text processing field labeling

Text Processing – Field Labeling

A powerful car bomb exploded today in Baghdadinside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington.

The blast came only two days after another car bomb exploded in a crowdedstreet in Mosul in the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Together with the shooting in Najaf three weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq ….


Text processing event segmentation

Text Processing - Event Segmentation

Event 1

  • A powerful car bomb exploded today in Baghdadinside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington. The blast came …

Event 2

  • only two days after another car bomb exploded in a crowdedstreet in Mosulin the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda.

Event 3

  • Together with the shooting in Najafthree weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level.

The bombing today in the capital of Iraq ….

Event 1


Text processing joint task

Text Processing – Joint Task


Machine translation

Machine Translation


Textual entailment

T1  H1

T1

T1

“At the end of the year, all solid companies pay dividends.”

“At the end of the year, all solid companies pay dividends.”

H1

H2

“At the end of the year, all solid insurance companies pay dividends.”

“At the end of the year, all solid companies pay cash dividends.”

T1  H2

Textual Entailment


Textual entailment1

Textual Entailment

  • Things can be much more challenging …..

Context Sensitive Paraphrasing:

  • Can speak replace command?

  • The general commanded his troops.

  • The general spoke to his troops.

  • The soloist commandedattention.

  • The soloist spoke to attention.


Text summarization

Text Summarization


Dialog systems

Dialog Systems


Another branch computational models in psycholinguistics

Another Branch - Computational Models in Psycholinguistics


This course

This Course

  • This course will focus on the basic NLP tasks:

    • Language modeling

    • Tagging tasks (sequence learning)

    • Syntax (as the prototype of complex structure learning)

    • Semantics (with an emphasize on lexical semantics)

  • We will take a data-driven, machine learning based approach


This course1

This Course

  • Why do we take this approach:

    • Allows us to develop a principled model based approach to NLP

    • Allows us to develop the fundamental machine learning tools of NLP

      • Particularly, allows us to learn about predicting structure

    • Focuses on the core of NLP – the models we will develop make NLP applications possible


Talk outline5

Talk Outline

  • Language and Language Technology

  • Language as a test case for scientific modeling

  • What is NLP ?

  • Basic NLP tasks (focus of this course)

  • NLP Applications

  • Why is NLP hard (ambiguity)


Ambiguity in language

Ambiguity in Language


Ambiguity in language1

Ambiguity in Language


Ambiguity in language2

Ambiguity in Language


Ambiguity in language3

Ambiguity in Language


Ambiguity in language4

Ambiguity in Language


Ambiguity in language5

Ambiguity in Language


Ambiguity in language6

Ambiguity in Language


Journals conferences and r esources

Journals, Conferences and Resources

  • http://aclweb.org/anthology//

  • Association for Computational Linguistics (ACL)

  • North American Association for Computational Linguistics (NAACL)

  • International Conference on Computational Linguistics (COLING)

  • Empirical Methods in Natural Language Processing (EMNLP)

  • Conference on Computational Natural Language Learning (CoNLL)

  • Transactions of the ACL (TACL)

  • Computational Linguistics


  • Login