1 / 8

NATURAL LANGUAGE PROCESSING (ALMOST) FROM SCRATCH

Ronan Collobert Jason Weston Leon Bottou Michael Karlen Koray Kavukcouglu Pavel Kuksa. NATURAL LANGUAGE PROCESSING (ALMOST) FROM SCRATCH. INTRODUCTION. Common Approaches in NLP Using Task-specific features Knowledge injection about structure of data Expertise from Linguists

nerita
Download Presentation

NATURAL LANGUAGE PROCESSING (ALMOST) FROM SCRATCH

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ronan Collobert Jason Weston Leon Bottou Michael Karlen KorayKavukcouglu PavelKuksa NATURAL LANGUAGE PROCESSING (ALMOST) FROM SCRATCH

  2. INTRODUCTION • Common Approaches in NLP • Using Task-specific features • Knowledge injection about structure of data • Expertise from Linguists • Approach used in the paper • No task-specific feature engineering • Minimal prior Knowledge

  3. Common Benchmark Datasets • Part-Of-Speech (POS) Tagging • Syntactic Parsing • Chunking • Shallow Parsing • Named Entity Recognition • Person, Location etc. • Semantic Role Labelling

  4. The Network • Words to Feature Vectors • Look up Table • Random initialization vs Unsupervised Pre-training • Extending to any Discrete Features • Extracting Higher level Features from Word Feature Vectors • Window Approach • Sentence Approach

  5. Window Approach Network

  6. Sentence Approach Network

  7. Training Schemes • Word-Level Log Likelihood • Only words are taken independently for optimizing the weights • Sentence-Level Log Likelihood • Optimization function takes into account all the tags as well as transitions between tags • Stochastic Gradient • Standard Optimization Algorithm

  8. Results

More Related