Code QuizIntermediate

Sampling From a Stochastic Policy

Spot the inverted comparison that breaks inverse-CDF action sampling from a policy distribution.

Codepython

import random

def sample_action(probs):
    # probs is a list of action probabilities summing to 1.0
    r = random.random()  # uniform in [0, 1)
    cumulative = 0.0
    for action, p in enumerate(probs):
        cumulative += p
        if r > cumulative:
            return action
    return len(probs) - 1

# Example: probs = [0.7, 0.2, 0.1]
print(sample_action([0.7, 0.2, 0.1]))

This function is meant to sample an action according to its probability, but the sampling is biased. What is the bug?

Next →Q-Table Initialization and Indexing