</>SkillRecallRECALL WHAT YOU KNOW
Explore Library
AI FundamentalsAI Infrastructure and MLOps

10 items

1

Dynamic Quantization Before Inference

Code Quiz
2

Model Packaging Dependencies and requirements

Code Quiz
3

CI/CD Pipeline Stage Ordering for ML

Code Quiz
4

Canary and Shadow Deployment Routing

Code Quiz
5

Containerizing a Model with Dockerfile

Code Quiz
6

Data Drift Detection Computation

Code Quiz
7

Model Performance Monitoring in Production

Code Quiz
8

Concept Drift Monitoring Logic

Code Quiz
9

Batch vs Online Inference Pattern Selection

Code Quiz
10

GPU/TPU Acceleration and Resource Management

Slides / Video
Code QuizAdvanced

Dynamic Quantization Before Inference

Spot why calling eval() on the wrong object leaves the quantized model in the wrong mode.

Codepython
import torch
from torch.quantization import quantize_dynamic

model = load_trained_model()  # a trained nn.Module

# Quantize Linear layers to int8 for faster CPU inference
quantized = quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)

model.eval()  # switch to inference mode

with torch.no_grad():
    output = quantized(sample_input)

What is the bug in this code?