Natural Language Processing with Transformers in Python
- Description
- Curriculum
- FAQ
- Reviews
Transformer models are the de-facto standard in modern NLP. They have proven themselves as the most expressive, powerful models for language by a large margin, beating all major language-based benchmarks time and time again.
In this course, we learn all you need to know to get started with building cutting-edge performance NLP applications using transformer models like Google AI’s BERT, or Facebook AI’s DPR.
We cover several key NLP frameworks including:
-
HuggingFace’s Transformers
-
TensorFlow 2
-
PyTorch
-
spaCy
-
NLTK
-
Flair
And learn how to apply transformers to some of the most popular NLP use-cases:
-
Language classification/sentiment analysis
-
Named entity recognition (NER)
-
Question and Answering
-
Similarity/comparative learning
Throughout each of these use-cases we work through a variety of examples to ensure that what, how, and why transformers are so important. Alongside these sections we also work through two full-size NLP projects, one for sentiment analysis of financial Reddit data, and another covering a fully-fledged open domain question-answering application.
All of this is supported by several other sections that encourage us to learn how to better design, implement, and measure the performance of our models, such as:
-
History of NLP and where transformers come from
-
Common preprocessing techniques for NLP
-
The theory behind transformers
-
How to fine-tune transformers
We cover all this and more, I look forward to seeing you in the course!
-
1
IntroductionA brief introduction to the course, and how to get the most out of it.
-
2
Course Overview
An overview of everything we'll be covering in this course.
-
3
Environment Setup
How to setup a local Python environment that aligns to the environment used throughout the course.
-
4
CUDA Setup
How to setup CUDA for CUDA enabled GPUs.
-
15
Stopwords
Here we'll start with our first NLP preprocessing technique, how we can use stopwords.
-
16
Tokens Introduction
In the first part of exploration of tokens in NLP, we'll look at word, character, punctuation, part-of-word tokens and more.
-
17
Model-Specific Special Tokens
In the second part of tokens in NLP, we'll look at model-specific special tokens.
-
18
Stemming
We take a look and the Porter and Lancaster stemmers.
-
19
Lemmatization
Here we take a look at reducing words to their lemma roots.
-
20
Unicode Normalization - Canonical and Compatibility Equivalence
Here we will introduce Unicode Normalization and the two forms of equivalence, canonical and compatibility.
-
21
Unicode Normalization - Composition and Decomposition
Here we take a look at the two different directions in Unicode Normalization, composition, and decomposition.
-
22
Unicode Normalization - NFD and NFC
We'll move onto applying Unicode Normalization in Python with both NFD and NFC forms.
-
23
Unicode Normalization - NFKD and NFKC
In the final Unicode Normalization session, we'll learn about and implement NFKD and NFKC forms.
-
54
Open Domain and Reading Comprehension
An introduction to the two modes of Q&A, open domain (OD) and reading comprehension (RC).
-
55
Retrievers, Readers, and Generators
An introduction to the three key model types we will be using in Q&A, retrievers, readers, and generators.
-
56
Intro to SQuAD 2.0
We introduce the SQuAD Q&A dataset.
-
57
Processing SQuAD Training Data
How we process the SQuAD data to be more friendly for our use-case.
-
58
(Optional) Processing SQuAD Training Data with Match-Case
We take a look at refactoring our SQuAD processing data using pattern matching syntax introduced in Python 3.10.
-
59
Processing SQuAD Dev Data -
60
Our First Q&A Model
We put together our first Q&A model.
-
61
Q&A Performance With Exact Match (EM)
-
62
ROUGE in Python
Learn how to implement ROUGE scores using Python.
-
63
Applying ROUGE to Q&A
We take a look at applying ROUGE performance metrics to our first Q&A model.
-
64
Recall, Precision and F1
We work through the intuition and mathematics behind ROUGE-N.
-
65
Longest Common Subsequence (LCS)
We work through the intuition and mathematics behind ROUGE-L.
-
66
Q&A Performance With ROUGE
We introduce the ROUGE metrics.
-
67
Intro to Retriever-Reader and Haystack
A quick introduction to the retriever-reader architecture and Haystack.
-
68
What is Elasticsearch?
An quick overview of how Elasticsearch works, and why we use it.
-
69
Elasticsearch Setup (Windows)
-
70
Elasticsearch Setup (Linux)
-
71
Elasticsearch in Haystack
We introduce Elasticsearch via Haystack.
-
72
Sparse Retrievers
-
73
Cleaning the Index
-
74
Implementing a BM25 Retriever
-
75
What is FAISS?
-
76
FAISS in Haystack
-
77
What is DPR?
-
78
The DPR Architecture
-
79
Retriever-Reader Stack
