Explore Library
Code QuizIntermediate

Mapping Tokens to Integer IDs

Spot the bug in building a token-to-ID vocabulary mapping.

Codepython
tokens = ["the", "cat", "sat", "the", "cat"]
word2id = {}
for i, token in enumerate(tokens):
    word2id[token] = i

print(word2id)

What is the bug in this token-to-ID mapping?