Code QuizIntermediate

TD Error Computation

Spot the sign mistake when computing the temporal-difference error.

Codepython

def td_error(reward, gamma, V, state, next_state):
    # TD target minus current estimate
    target = reward + gamma * V[next_state]
    error = V[state] - target
    return error

What is the bug in this TD error computation?