Explore Library
Code QuizIntermediate

Estimating State Value

Find the mistake in estimating V(s) from sampled returns.

Codejavascript
// V(s) = expected return starting from state s
function stateValue(returnsFromS) {
  let sum = 0;
  for (const g of returnsFromS) {
    sum += g;
  }
  return sum;
}

What is the bug in this state-value estimate?