AI FundamentalsModel Evaluation and Validation

109 items

1

Ranking Metrics on Imbalanced Data

Code Quiz
2

PR vs ROC on Imbalanced Fraud Data

Quiz
3

PR vs ROC on Imbalanced Data

Flashcard
4

Metrics for Imbalanced Data

Slides / Video
5

Cross-Validation on Time-Series Data

Code Quiz
6

Metric & Validation Strategy Selection

Quiz
7

Choosing Metrics and Validation Strategy

Flashcard
8

Choosing Metrics and Validation

Slides / Video
9

Cross-Validation Data Leakage Bug

Code Quiz
10

Cross-Validation on Imbalanced Data

Quiz
11

Metrics & Cross-Validation

Flashcard
12

Evaluation Metrics & Cross-Validation

Slides / Video
13

Data Leakage in Model Evaluation

Code Quiz
14

Recognizing Overfitting

Quiz
15

Overfitting & Bias-Variance Tradeoff

Slides / Video
16

Overfitting & Bias-Variance Tradeoff

Flashcard
17

Core Evaluation Metrics at a Glance

Slides / Video
18

Splits and Cross-Validation Basics

Slides / Video
19

Common Evaluation Metrics Overview

Flashcard
20

Data Splits and Cross-Validation Basics

Flashcard
21

Learning Curves for Model Diagnosis

Slides / Video
22

Bias-Variance Tradeoff

Slides / Video
23

Stratified Cross-Validation for Imbalanced Data

Slides / Video
24

K-Fold Cross-Validation

Slides / Video
25

Train/Validation/Test Split Purpose

Slides / Video
26

Regression Metrics: MAE, MSE, RMSE, R2

Slides / Video
27

ROC Curve and AUC

Slides / Video
28

Precision-Recall Tradeoff

Slides / Video
29

Precision, Recall, and F1 Score

Slides / Video
30

Confusion Matrix Structure and Interpretation

Slides / Video
31

Why Accuracy Alone Is Insufficient

Slides / Video
32

Choosing the Right Metric

Slides / Video
33

Baseline Models for Comparison

Slides / Video
34

Evaluating Models on Imbalanced Data

Slides / Video
35

Data Leakage and How to Avoid It

Slides / Video
36

Hyperparameter Tuning with Validation Set

Slides / Video
37

Grid Search Parameter Grid

Code Quiz
38

Time-Series Split Ordering

Code Quiz
39

Bias-Variance Diagnosis

Code Quiz
40

Nested Cross-Validation Estimate

Code Quiz
41

Leave-One-Out Setup

Code Quiz
42

Overfitting Gap From Curves

Code Quiz
43

Underfitting Detection Logic

Code Quiz
44

Three-Way Split Proportions

Code Quiz
45

Precision-Recall Curve Use

Quiz
46

Macro vs Micro vs Weighted Averaging

Quiz
47

Overfitting to the Test Set

Quiz
48

Stratified Cross-Validation

Quiz
49

Cross-Validation for Small Datasets

Quiz
50

Choosing the Right Metric

Quiz
51

Hyperparameter Tuning with Validation

Quiz
52

F1 Score Basics

Quiz
53

Choosing a Classification Threshold

Quiz
54

Baseline Models

Quiz
55

Understanding Recall

Quiz
56

Reading a Confusion Matrix

Quiz
57

K-Fold Cross-Validation

Quiz
58

Specificity and False Positive Rate

Quiz
59

Holdout vs Cross-Validation

Quiz
60

Definition of Precision

Quiz
61

R-squared vs Adjusted R-squared

Quiz
62

Detecting Overfitting

Quiz
63

Log Loss as a Metric

Quiz
64

Metrics for Imbalanced Classification

Quiz
65

Bias-Variance Tradeoff

Quiz
66

ROC Curve and AUC

Quiz
67

Accuracy on Imbalanced Data

Quiz
68

Leave-One-Out Cross-Validation

Quiz
69

MAE vs MSE vs RMSE

Quiz
70

Baseline Dummy Classifier Comparison

Code Quiz
71

Aggregating Cross-Validation Scores

Code Quiz
72

Computing MAE

Code Quiz
73

Macro vs Weighted F1 Average

Code Quiz
74

R-squared Interpretation

Code Quiz
75

Accuracy on Imbalanced Data

Code Quiz
76

Thresholding Probabilities to Labels

Code Quiz
77

K-Fold Cross-Validation Setup

Code Quiz
78

Stratified K-Fold for Balance

Code Quiz
79

Computing RMSE

Code Quiz
80

Accuracy From a Confusion Matrix

Code Quiz
81

ROC Curve and AUC Interpretation

Code Quiz
82

Specificity and False Positive Rate

Code Quiz
83

F1 Score Computation Bug

Code Quiz
84

Precision-Recall Curve Baseline

Code Quiz
85

Reading a Confusion Matrix Layout

Code Quiz
86

Recall (Sensitivity) Calculation Bug

Code Quiz
87

Precision Calculation Bug

Code Quiz
88

Bias-Variance Tradeoff

Flashcard
89

Classification Threshold Selection

Flashcard
90

Metric Selection Based on Problem Type

Flashcard
91

Regression Metrics: MAE

Flashcard
92

Evaluation on Imbalanced Datasets

Flashcard
93

Regression Metrics: MSE and RMSE

Flashcard
94

Learning Curves for Diagnosis

Flashcard
95

Baseline Model for Comparison

Flashcard
96

R-Squared (Coefficient of Determination)

Flashcard
97

Hyperparameter Tuning with Validation Set

Flashcard
98

Precision-Recall Tradeoff

Flashcard
99

K-Fold Cross-Validation

Flashcard
100

Stratified K-Fold Cross-Validation

Flashcard
101

F1 Score

Flashcard
102

Specificity

Flashcard
103

Precision

Flashcard
104

Train/Validation/Test Three-Way Split

Flashcard
105

Holdout Validation vs Cross-Validation

Flashcard
106

ROC Curve and AUC

Flashcard
107

Leave-One-Out Cross-Validation (LOOCV)

Flashcard
108

Recall / Sensitivity

Flashcard
109

Confusion Matrix

Flashcard
QuizIntermediate

Bias-Variance Tradeoff

Learn what high bias and high variance mean for a model.

A model with high bias and low variance typically exhibits which behavior?