Questioning the Effectiveness of Person Re-Identification Datasets

14 May 2026, by Viktoria Wrobel

Photo: base.camp

Surveillance systems that match and track people across cameras rely on ap-
pearance encoders, which are commonly trained on Person Re-Identification
(ReID) datasets. These datasets are ethically problematic: they contain im-
ages of people captured without meaningful consent and have been used by
organizations conducting mass surveillance. Despite these concerns, their
practical benefit over general-purpose encoders remains assumed rather than
empirically tested. This project investigates whether ReID-finetuned en-
coders actually improve multi-object tracking performance compared to en-
coders trained with identical methods on non-surveillance data. Through
controlled tracking experiments and probing analysis of learned representa-
tions, we test the hypothesis that ReID encoders learn dataset-specific biases
rather than robust identity features. If ReID finetuning provides no measur-
able benefit, the privacy violations inherent in surveillance dataset collection
lack empirical justification.