Lecture 1: Introduction to NLP and Deep Learning
What is Natural Language Processing
Intersecting of
let computer process or “understand” NL to perform tasks
- Performing tasks
- question answering
Fully understanding and representing the meaning of language is a difficult goal.
NLP Levels
speech text
Phonetic/Phonological Analysis OCR/Tokenization
Morphological analysis (单词结构的形态分析)
Syntactic analysis (句法分析)
Semantic Interpertation (语义理解)
Discourse Processing (上下文，语篇处理)
(A tiny sample of) NLP Applications
- Spell checking, keyword search, finding synonyms
- Extracting information from websites
- Classifying
- Machine translation
- Spoken dialog systems
- Complex question answering
NLP in industry … is taking off
- Search (written and spoken) like Spell Check
- Online advertisement matching
- Automated/assisted translation
- Sentiment analysis for marketing of finance/trading
- Speech recognition
- Chatbots/Dialog agents
What’s special about human language
a system specifically constructed to convey the speaker/writer’s meaning
- deliberate communication
- little kids can quickly learn
and A human language is a discrete/symbolic/categorical signaling system:
- symbolic signaling system
- with very minor exceptions for expressive signaling
- symbols are not just an invention of logic/classical AI
The categorical symbols of a language can be encoded as a signal for communication in several ways:
- Sound
- Gesture
- Images(writing)
A human language is a symbolic/categorical signaling system
a brain encoding appears to be a continuous pattern of activation, and the symbols are transmitted via continuous signals of sound/vision.
What’s Deep Learning(DL)
Deep learning is a subfield of machine learning. blablablabla…
- Representation Learning attempts to auto matically learn good features or representations.
- Deep Learning algorithms attempt to learn (multiple levels of) representation and an output
- From “raw” inpus x
强调类似于“端到端”的模式
On the history of and term “Deep Learning”
focus on neural networks blabla…
Reasons for Exploring Deep Learning
- Manually designed features are often over-specified, incomplete and take a long time to design and validate
- Lerned Features are easy to adapt, fast to learn.
- Deep learning provides a very flexible, (almost?) universal, learnable framework for representing world, visual and linguistic information.
- Deep Learning can learn unsupervised (from raw text) and supervised (with specific labels like positive/negative)
good performing and CPU/GPUs quickly processing and more and more data.
Deep Learning for Speech
- starts with speech recognition
- ImageNet vision compitation
Why is NLP hard
- Complexity in representing, learning and using linguistic/situational/world/world/visual knowledge
- Human languages are ambiguous (unlick programming and other formal languages)
- Human language interpretation depends on real world, common sense, and contextual knowledge
We’re these unbelievably complicated beings drifting through a void, trying in vain to connect with one another by blindly flinging words out into the darknees. Every trace of phrasing, and spelling and tone and timing carries countless signals and contexts and subtexts and more. And every listener interprets these signals in their own way. Language isn’t a formal system of language, It’s glorious chaos. You can never know for sure what any words will mean to anyone. All you can do is try to get better at guessing how your words affect people. So, you have a chance of finding the ones that will make them feel something like you want them to feel. Everything else is pointless.
Deep NLP = Deep Learning + NLP
using representation learning and deep learning methods to solve NLP problems.
Do what
- Word similarities
- Representations of NLP Levels: Morphology
- NLP Tools: Parsing for sentence structure
- RepreSemantions of NLP Levels: Semantics
- NLP Applications: Sentiment Analysis
- Question Answering
- Dialogue agents / Response Generation
- Machine Translation
- Neural Machine Translation
Conclusion:Representation for all levels? Vectors
Reference
- EE information theory
- Chen Danqi
