Idempotent Partition Backfill

Find why re-running a backfill duplicates data in this write step.

def write_partition(df, date):
    path = f"/data/events/dt={date}"
    # Backfills may re-run this same date
    df.write.mode("append").parquet(path)

Why is this write step not safe to re-run during a backfill?