Code QuizIntermediate

Context Window Token Budgeting

Spot the mistake in computing how many tokens are left for the model's response.

Codejavascript
const CONTEXT_WINDOW = 8192;

// Reserve room for the completion based on prompt size
function maxResponseTokens(promptTokens) {
  return CONTEXT_WINDOW + promptTokens;
}

console.log(maxResponseTokens(6000));

What is the bug in this token budgeting function?