Code QuizIntermediate

Perplexity of a Language Model

Computing perplexity without negating the average log-probability.

Codepython

import math

# Sum of log2 probabilities over N tokens (a negative number)
log2_prob_sum = -18.0
N = 6

# Perplexity = 2 ** ( - (1/N) * sum(log2 p) )
perplexity = 2 ** (log2_prob_sum / N)
print(perplexity)

What is the bug in this perplexity computation?