Skip to content

Natural Language Processing (NLP) - Notes

Overview

NLP is a computer science, artificial Intelligence subsets which deals with Human Language.

Application

  • Sentimental Analysis: (ex: fb emojis)
  • Chatbot (costumer assistance)
  • Speech Recognition (voice assistance, like windows cortana)
  • Machine Translation (google translate)
  • Spell Checking
  • Information Extraction
  • Keyword Searching
  • Advertisement

Components of NLP

Natural Language Understanding (NLU) - Mapping input to useful representations - Analyzing different aspects of the language - Ambiguity lexical - Ambiguitysyntactic - Ambiguity referential

Natural Language Generation (NLG) - Text planning - Sentence planning - Text realization

TEXT MINING: analysation of information from natural language text.

NLP Pipeline

alt text

Src: Basic Steps In Natural Language Processing Pipeline

NLP Techniques

  1. SYNTACTIC ANALYSIS : understanding the grammar of the text
  2. Segmentation : split document into contituent units (one or more sentences)
  3. Tokenization : split a phrase into a small part od token
  4. Stemming : normalize words into its base form or root form
  5. Lemmatization: morphological analyse of the phrase (based on dictionary : meaning and synonym)
  6. POS tagging : classify a part of the speech : verb, adj, noun
  7. Named Entity Recognition (NER): classify a group of word in a group : movie, monetary value, organizatio, location, quantities, person
  8. Chunking: picking individual pieces of informations and grouping them into bigger Pieces

  9. Parsing(tree): grammatical analysis of the sentence

  10. SEMANTIC ANALYSIS : understanding the literal meaning of the text.

  11. Text Correction
  12. Text generation
  13. Machine Translation
  14. Word sense disambiguation

  15. Vector/Embeddings (high-dimensional vectors)

  16. PRAGMATIC ANALYSIS: understanding of what the text is trying to achieve

Content Generation

  • Next Sentence Prediction (NSP) : Check if the sentence B, follows the sentence A.

Tools: - Deep Learning - GenAI and LLM - Multimodal AI - RAG (Retrieval-Augmented Generation) - Embeddings (high-dimensional vectors)

Machine Translation

Src: Study tonight

Large Language Models (LLMs)

Feature NLP LLM
Focus Broad range of techniques for specific tasks Specific type of NLP model, broad capabilities
Strength Accuracy in specific tasks Versatility, open ended tasks
Underlying Techniques Rule-based, Machine Learning, Deep Learning Deep Learning (Transformers)
Applications Chatbots (specific functions), sentiment analysis, named entity recognition Creative text generation, summarization, open ended chatbots
  • More about LLMs here.

Tools & Frameworks

References

Wikipedia

  • Natural Language Processing: https://en.wikipedia.org/wiki/Natural_language_processing
  • MT: https://en.wikipedia.org/wiki/Machine_translation
  • Neural MT: https://en.wikipedia.org/wiki/Neural_machine_translation
  • NLP: https://en.wikipedia.org/wiki/Natural_language_processing
  • semantic wiki: https://en.wikipedia.org/wiki/Semantics

Lecture and Online Courses: - ibm: - What is NLP (Natural Language Processing)? - ibm nlp vs nlu vs nlg: https://www.youtube.com/watch?v=1I6bQ12VxV0&t=12s - Machine Translation - ibm lstm: https://www.youtube.com/watch?v=b61DPVFX03I - npl vs nlu: https://www.kdnuggets.com/2019/07/nlp-vs-nlu-understanding-language-processing.html - Google Research: https://research.google/research-areas/natural-language-processing/ - NLP for Developers - Full course Rasa

Frameworks: - https://odsc.medium.com/10-notable-frameworks-for-nlp-ce8c4196bfd6 - NLP w/ python & nltk: - https://www.youtube.com/watch?v=U8m5ug9Q54M
- https://www.youtube.com/watch?v=05ONoGfmKvA&t=0s

MT: - https://www.studytonight.com/post/different-machine-translation-models-in-nlp

Word Embedding and Vector Space Model - Word embedding - Vector Space Models

NLP vs DL vs GenAI vs LLM Tools: - Unleashing the Power of Langchain: Advancing NLP with Chained Language Models - NLP vs LLM: A Comprehensive Guide to Understanding Key Differences - RAG vs semantic search, embedding, vector... - RAG vs Finetuning — Which Is the Best Tool to Boost Your LLM Application?

Projects: - Self-Driving Taxi Chatbot with Pytorch

Research papers: - Attention is all you need (paper): https://research.google/pubs/pub46201/ - analyticsindiamag : https://analyticsindiamag.com/10-must-read-technical-papers-on-nlp-for-2020/ - slator : https://slator.com/here-are-the-best-natural-language-processing-papers-from-acl-2022/ - paperdigest: https://www.paperdigest.org/category/nlp/ - paper with code: https://paperswithcode.com/task/language-modelling