AI Quality Workshop: How to Test and Debug ML Models
- Description
- Curriculum
- FAQ
- Reviews
Want to skill up your ability to test and debug machine learning models? Ready to be a powerful contributor to the AI era, the next great wave in software and technology?
Get taught by leading instructors who have previously taught at Carnegie Mellon University and Stanford University, and who have provided training to thousands of students from around the globe, including hot startups and major global corporations:
-
You will learn the analytics that you need to drive model performance
-
You will understand how to create an automated test harness for easier, more effective ML testing
-
You will learn why AI explainability is the key to understanding the key mechanics of your model and to rapid debugging
-
Understand what Shapley Values are, why they are so important, and how to make the most of them
-
You will be able to identify the types of drift that can derail model performance
-
You will learn how to debug model performance challenges
-
You will be able to understand how to evaluate model fairness and identify when bias is occurring – and then address it
-
You will get access to some of the most powerful ML testing and debugging software tools available, for FREE
(after signing up for the course, terms and conditions apply)Testimonials from the live, virtual version of the course:
-
“This is what you would pay thousand of dollars for at a university.” – Mike
-
“Excellent course!!! Super thanks to Professor Datta, Josh, Arri, and Rick!! :D” – Trevia
-
“Thank you so very much. I learned a ton. Great job!” – K. M.
-
“Fantastic series. Great explanations and great product. Thank you.” – Santosh
-
“Thank you everyone to make this course available… wonderful sessions!” – Chris
-
1
Welcome - what you'll get from this courseWelcome to the course! We're glad that you're here and excited that you're going to get started on a path that will help you to build more effective and fairer ML models. Professor Datta explains what you can expect on this journey.
-
2
How to set up your free TruEra access at app.truera.net/signup
As part of this course, you will be using a "sandbox" environment in TruEra that is pre-loaded with models and data. As a member of this course, you have FREE access to TruEra. This video walks you through how to get started. You can sign up at https://app.truera.net/signup
Also, please sign up for the AI Quality Forum - this is the best place to get questions answered and to meet a community of peers.
Note: the free access also gives you the ability to use TruEra with your own models and data, although that is not a part of the course.
Josh Reini, data scientist, shows you how to get started. He will also be providing product demonstrations throughout the course. You can also meet him virtually in the AI Quality Forum. -
3
How to use Google Colab for TruEra
We will be using GitHub and Google Colab throughout the course. This is how you can get started.
-
4
Introduction to ML Testing
Welcome to the start of the course learning content! First, we will be doing an overview of ML Testing, which shows how the analyses that we will introduce you to in-depth later can all pull together into a powerful testing environment for you to drive model performance.
-
5
Running and Interpreting Tests
-
6
Creating New Tests
-
7
Introduction to ML Explainability
ML Explainability is a core concept and analytical skill. It is the engine driving your ability to evaluate, debug, and iterate on your models. This section will give you a solid grounding in what explainbility is, why it is important, and how to calculate and leverage it.
-
8
Overview of Feature Importance Methods
-
9
Shapley Values - Query Definition
-
10
Shapley Values - Comparing Model Outputs
-
11
Shapley Values - Dealing with Feature Interactions
-
12
Shapley Values - Summarization
-
13
Overview - Gradient Based Explanations for Computer Vision
-
14
Design - Gradient-Based Explanations for Computer Vision
-
15
Evaluation - Gradient-Based Explanations for Computer Vision
-
16
Hands-On Learning - Explainability
-
17
Quiz - Explainability -
18
Demonstration - Global and Local Explainability Analysis
-
19
Introduction to Drift
Understanding drift - what it is, how it happens, and how to address it - will be key to your ability to debug models both in development and after they go into production (live use). Shayak Sen teaches this section of the course.
-
20
Sources of Drift: Why Does Drift Happen?
-
21
Drift Quiz #1
-
22
Identifying Drift: Metrics
-
23
Identifying Drift: Challenges
-
24
How to Mitigate Drift
-
25
Hands-on Learning: Drift
-
26
Drift Quiz #2
-
27
Demonstration - Going from the Model Summary to Drift Analytics
-
28
Introduction to ML Performance Debugging
Now that you can identify that your model is having issues and understand drift, how do you debug your models? This section provides an overview of how you can diagnose and address model issues.
-
29
ML Peformance Debugging Methodology
-
30
ML Performance Metrics - Classification
-
31
ML Performance Metrics - Regression
-
32
Narrowing Down the Scope of ML Performance Issues
-
33
Hands-On Learning: Performance Debugging
-
34
Quiz: Performance Debugging
-
35
Demonstration - Performance Debugging
