</>SkillRecallRECALL WHAT YOU KNOW
Explore Library
AI FundamentalsModel Evaluation and Validation

20 items

1

Ranking Metrics on Imbalanced Data

Code Quiz
2

PR vs ROC on Imbalanced Fraud Data

Quiz
3

PR vs ROC on Imbalanced Data

Flashcard
4

Metrics for Imbalanced Data

Slides / Video
5

Cross-Validation on Time-Series Data

Code Quiz
6

Metric & Validation Strategy Selection

Quiz
7

Choosing Metrics and Validation Strategy

Flashcard
8

Choosing Metrics and Validation

Slides / Video
9

Cross-Validation Data Leakage Bug

Code Quiz
10

Cross-Validation on Imbalanced Data

Quiz
11

Metrics & Cross-Validation

Flashcard
12

Evaluation Metrics & Cross-Validation

Slides / Video
13

Data Leakage in Model Evaluation

Code Quiz
14

Recognizing Overfitting

Quiz
15

Overfitting & Bias-Variance Tradeoff

Slides / Video
16

Overfitting & Bias-Variance Tradeoff

Flashcard
17

Core Evaluation Metrics at a Glance

Slides / Video
18

Splits and Cross-Validation Basics

Slides / Video
19

Common Evaluation Metrics Overview

Flashcard
20

Data Splits and Cross-Validation Basics

Flashcard
Code Quiz

Cross-Validation on Time-Series Data

Spot the validation-strategy bug when evaluating a model on time-ordered data.

Codepython
from sklearn.model_selection import KFold, cross_val_score
from sklearn.linear_model import LinearRegression
import numpy as np

# X is ordered by timestamp (oldest -> newest), y is next-day price
X = np.load('daily_features.npy')
y = np.load('next_day_price.npy')

model = LinearRegression()
cv = KFold(n_splits=5, shuffle=True, random_state=42)

scores = cross_val_score(model, X, y, cv=cv, scoring='r2')
print('Mean R2:', scores.mean())

This forecasting evaluation gives overly optimistic scores. What is the bug?