Summary - F.748.33 (10/2024) - Metrics and evaluation methods for image-based re-identification algorithm

Recommendation ITU-T F.748.33 provides a benchmark framework, metrics, requirements for testing datasets and evaluation process of image-based re-identification (Re-ID) algorithms. Image-based ReID is a critical task in machine vision systems with a wide range of applications in video surveillance system, intelligent transportation, etc. Owing to the advancement of deep learning, the image-based Re-ID algorithms have achieved impressive progress in academia. However, these state-of-art algorithms may have a significant performance drop when deployed in practical unconstrained scenarios. The existing evaluation method which mainly rely on pre-prepared open-source datasets cannot reflect the real performance of the algorithms in practical applications. It is crucial to have a unified, systematic, and fair evaluation method to comprehensively compare and analyse the diverse algorithms performance for both academia and industry.