1 / 52

Natural Language Processing Lecture 1 : Introduction to NLP

Natural Language Processing Lecture 1 : Introduction to NLP. Winter 2014-15 Lecturer: Prof. Roi Reichart. Course Information. Lecture: Roi Reichart. TA: Ira Leviant Class: Thursday 11:30 - 14:30, TA: Thursday 10:30 - 11:30. Course Prerequisites.

leigh-chen
Download Presentation

Natural Language Processing Lecture 1 : Introduction to NLP

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Natural Language ProcessingLecture 1: Introduction to NLP Winter 2014-15 Lecturer: Prof. Roi Reichart

  2. Course Information • Lecture: Roi Reichart. TA: Ira Leviant • Class: Thursday 11:30 - 14:30, TA: Thursday 10:30 - 11:30

  3. Course Prerequisites • Basic Linear Algebra, Probability, Algorithms • Machine learning will help (but some of it will be reviewed in class and TA sessions) • Programming skills

  4. Course Grading • Class and TA attendance – 5% (mandatory, contact me if you cannot attend) • Student lecture – 20% (last hour every week) • 40% - four home work assignments (one programming question plus a few theoretical ones) • 35% - final project (?)

  5. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

  6. Natural Language Processing (NLP)

  7. Natural Language Processing (NLP)

  8. Natural Language Processing (NLP) - Interdisciplinary

  9. Natural Language Processing (NLP) - Technology

  10. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

  11. Modeling Language learning and Processing The four components of scientific description of a phenomenon • Theory • Model • Parameter estimation, a.k.a learning • Prediction, a.k.a inference

  12. Modeling Language Learning and Processing The four components of scientific description of a phenomenon • Theory – many layers of language analysis are hidden (i.e. does not exist in the world) • Model • Parameter estimation, a.k.a learning • Prediction, a.k.a inference

  13. Modeling Language Learning and Processing

  14. Modeling Language Learning and Processing The four components of scientific description of a phenomenon • Theory – many layers of language analysis are hidden (i.e. does not exist in the world) • Model – since many of the layers are hidden there can be strong disagreement on their modeling • Parameter estimation, a.k.a learning • Prediction, a.k.a inference

  15. Modeling Language Learning and Processing The four components of scientific description of a phenomenon • Theory – many layers of language analysis are hidden (i.e. does not exist in the world) • Model – since many of the layers are hidden there can be strong disagreement on their modeling • Parameter estimation, a.k.a learning – as models contain hidden layers this can be complicated • Prediction, a.k.a inference – as models contain hidden layers this can be complicated

  16. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

  17. What is in NLP ? • Machine Learning and Optimization (especially structured prediction)

  18. What is in NLP ? • Machine Learning and optimization (especially structured prediction) • Statistical Modeling (parameter estimation, inference)

  19. What is in NLP ? • Machine Learning and optimization (especially structured prediction) • Statistical Modeling (parameter estimation, inference) • Linguistics

  20. What is in NLP ? • Machine Learning and optimization (especially structured prediction) • Statistical Modeling (parameter estimation, inference) • Linguistics • Cognitive Science

  21. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

  22. Language Processing

  23. Basic NLP Tasks • Basic input analysis (word segmentation):

  24. Basic NLP Tasks • Basic input analysis (word segmentation):

  25. Basic NLP Tasks • Word level analysis (morphological segmentation):

  26. Basic NLP Tasks • Sentence level analysis: Syntax, Part-of-Speech (POS) tagging: I gave the smaller balls to Itamar PRP V DT JJ N IN N PRP – Personal Pronoun JJ - Adjective V – Verb N – Noun DT – Determiner IN - Preposition

  27. Basic NLP Tasks • Sentence level analysis: Syntax, syntactic parsing: I gave the ball to Itamar PRP VBD DT NN TO NNP

  28. Basic NLP Tasks • Sentence level analysis: lexical semantics:

  29. Basic NLP Tasks • Word meaning (semantics):

  30. Basic NLP Tasks • Multilingual Processing:

  31. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

  32. Text Processing A powerful car bomb exploded today in Baghdad inside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington. The blast came only two days after another car bomb exploded in a crowdedstreet in Mosul in the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Together with the shooting in Najaf three weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq …

  33. Text Processing – Field Labeling A powerful car bomb exploded today in Baghdadinside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington. The blast came only two days after another car bomb exploded in a crowdedstreet in Mosul in the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Together with the shooting in Najaf three weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq ….

  34. Text Processing - Event Segmentation Event 1 • A powerful car bomb exploded today in Baghdadinside the holiest shite shrine. As many as 95 people were killed in the event, according to sources in Washington. The blast came … Event 2 • only two days after another car bomb exploded in a crowdedstreet in Mosulin the northern part of Iraq, killing 13 pedestrians, in an attack carried by Al Qaeda. Event 3 • Together with the shooting in Najafthree weeks ago that killed 15 American soldiers, violence seemed to spike to its highest level. The bombing today in the capital of Iraq …. Event 1

  35. Text Processing – Joint Task

  36. Machine Translation

  37. T1  H1 T1 T1 “At the end of the year, all solid companies pay dividends.” “At the end of the year, all solid companies pay dividends.” H1 H2 “At the end of the year, all solid insurance companies pay dividends.” “At the end of the year, all solid companies pay cash dividends.” T1  H2 Textual Entailment

  38. Textual Entailment • Things can be much more challenging ….. Context Sensitive Paraphrasing: • Can speak replace command? • The general commanded his troops. • The general spoke to his troops. • The soloist commandedattention. • The soloist spoke to attention.

  39. Text Summarization

  40. Dialog Systems

  41. Another Branch - Computational Models in Psycholinguistics

  42. This Course • This course will focus on the basic NLP tasks: • Language modeling • Tagging tasks (sequence learning) • Syntax (as the prototype of complex structure learning) • Semantics (with an emphasize on lexical semantics) • We will take a data-driven, machine learning based approach

  43. This Course • Why do we take this approach: • Allows us to develop a principled model based approach to NLP • Allows us to develop the fundamental machine learning tools of NLP • Particularly, allows us to learn about predicting structure • Focuses on the core of NLP – the models we will develop make NLP applications possible

  44. Talk Outline • Language and Language Technology • Language as a test case for scientific modeling • What is NLP ? • Basic NLP tasks (focus of this course) • NLP Applications • Why is NLP hard (ambiguity)

  45. Ambiguity in Language

  46. Ambiguity in Language

  47. Ambiguity in Language

  48. Ambiguity in Language

  49. Ambiguity in Language

  50. Ambiguity in Language

More Related