FRTE 1:1 Verification

These comments provided by Neurotechnology are based on NIST FRTE 1:1 Verification ongoing evaluation results reviewed on November 27, 2023.

What is FRTE 1:1 Verification evaluation?

NIST FRTE 1:1 Verification evaluates how well face recognition technologies work across various uses, from visa image checks to spotting duplicates in passports or identifying individuals in random photos. Eight testing scenarios range from simple dataset deduplication to real-life situations with cooperating and non-cooperating subjects. Algorithms' accuracy is the primary assessment metric, with additional metrics like matching speed and memory consumption. The datasets feature only one face per image.

See FRTE official page for more information.

The Face Recognition Technology Evaluation (FRTE) started in July, 2023, as a successor of the previous FRVT Ongoing evaluation of face verification algorithms. The FRTE 1:1 Verification track allows vendors to submit multiple algorithms once every three months, but the report takes into account only 2 latest submissions per vendor. Thus, only 539 algorithms are analyzed in the current results table from NIST.

Neurotechnology algorithm has been ranked in the top 3% of most accurate results for border control supervised (Visa Border, Border) and unsupervised (Border-Kiosk) scenarios among all submissions. The results for each scenario are presented below:

• FRTE 1:1 Visa Photos
• FRTE 1:1 Mugshot Photos
• FRTE 1:1 Mugshot Photos ΔT >12 YRS
• FRTE 1:1 Visa vs Border Photos
• FRTE 1:1 Visa vs Border Yaw ≥45° Photos
• FRTE 1:1 Border Photos
• FRTE 1:1 Border vs. Kiosk Photos

The comparisons were performed between all face templates from the VISA dataset.

Neurotechnology algorithm accuracy in this scenario was 0.31% FNMR at 0.0001% FMR. The most accurate contender showed 0.06% FNMR at the same FMR.

Scenario overview

• One image from the dataset was used to create one face template.
• All face templates were compared between each other.
• Each comparison score was obtained by comparing two different templates (simple one-to-one verification).
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Accuracy versus performance metrics

The charts below show the accuracy of all tested algorithms, with the highlighted dots corresponding to the Neurotechnology algorithm submissions.

The comparisons were performed between all face templates from the MUGSHOT dataset.

Neurotechnology algorithm accuracy in this scenario was 0.23% FNMR at 0.001% FMR. The most accurate contender showed 0.21% FNMR at the same FMR.

Scenario overview

• One image from the dataset was used to create one face template.
• All face templates were compared between each other.
• Each comparison score was obtained by comparing two different templates (simple one-to-one verification).
• Gender attribute was used to exclude comparisons between different gender subjects.

Accuracy versus performance metrics

The charts below show the accuracy of all tested algorithms, with the highlighted dots corresponding to the Neurotechnology algorithm submissions.

The test was performed using facial templates, generated from the MUGSHOT dataset. The comparisons were performed only between templates representing same or different persons with age difference of 12 or more years between them.

Neurotechnology algorithm accuracy in this scenario was 0.23% FNMR at 0.001% FMR. The most accurate contender showed 0.19% FNMR at the same FMR.

Scenario overview

• One image from the MUGSHOT dataset was used to create one face template.
• Age attribute was used to exclude template pairs, which represented same or different persons with age difference of less than 12 years at the capture moment, from the comparison.
• Gender attribute was used to exclude comparisons between different gender subjects.
• Each comparison score was obtained by comparing two different templates (simple one-to-one verification).

Accuracy versus performance metrics

The charts below show the accuracy of all tested algorithms, with the highlighted dots corresponding to the Neurotechnology algorithm submissions.

The test was performed by comparing high-quality images from the VISA dataset against lower quality images from the BORDER dataset. Both datasets include subjects from more than 100 countries, with specific imbalances due to visa issuance patterns and border-crossing demographics correspondingly.

Neurotechnology algorithm accuracy in this scenario was 0.25% FNMR at 0.0001% FMR. The most accurate contender showed 0.16% FNMR at the same FMR.

Scenario overview

• One image from Visa or Border datasets was used to create one face template.
• Gender attribute was used to exclude comparisons between different gender subjects.
• Each pre-filtered template from the Visa dataset was compared against each pre-filtered template from the Border dataset
• Each comparison score was obtained by comparing two different templates (simple one-to-one verification).

Accuracy versus performance metrics

The charts below show the accuracy of all tested algorithms, with the highlighted dots corresponding to the Neurotechnology algorithm submissions.

The test was performed by comparing high-quality images from the VISA dataset against lower quality images from the BORDER dataset. The comparisons were performed only between photos representing same or different persons with face yaw angle difference of 45 or more degrees between them.

Neurotechnology algorithm accuracy in this scenario was 1.24% FNMR at 0.0001% FMR. The most accurate contender showed 0.25% FNMR at the same FMR.

Scenario overview

• One image from Visa or Border datasets was used to create one face template.
• Gender attribute was used to exclude comparisons between different gender subjects.
• Face yaw angle attribute was used to filter face template pairs for achieving the required yaw angle difference of 45 or more degrees.
• Each pre-filtered template from the Visa dataset was compared against each pre-filtered template from the Border dataset
• Each comparison score was obtained by comparing two different templates (simple one-to-one verification).

Accuracy versus performance metrics

The charts below show the accuracy of all tested algorithms, with the highlighted dots corresponding to the Neurotechnology algorithm submissions.

The comparisons were performed between face templates from the MUGSHOT dataset, using gender attribute to exclude comparisons between different gender subjects.

Neurotechnology algorithm accuracy in this scenario was 0.47% FNMR at 0.0001% FMR. The most accurate contender showed 0.32% FNMR at the same FMR.

Scenario overview

• One image from the dataset was used to create one face template.
• Gender attribute was used to exclude comparisons between different gender subjects.
• All pre-filtered face templates were compared between each other.
• Each comparison score was obtained by comparing two different templates (simple one-to-one verification).

Accuracy versus performance metrics

The charts below show the accuracy of all tested algorithms, with the highlighted dots corresponding to the Neurotechnology algorithm submissions.

The test was performed by comparing higher quality images from the BORDER dataset against lower quality images from the KIOSK dataset. Both datasets include subjects from more than 100 countries, with specific imbalances due to visa issuance patterns and border-crossing demographics correspondingly.

The dataset is made from images. which were captured from cooperating subjects in a partially controlled environment. Images' quality varied due to an automated, non-supervised capture method.

Neurotechnology algorithm accuracy in this scenario was 5.11% FNMR at 0.001% FMR. The most accurate contender showed 3.94% FNMR at the same FMR.

• One image from each dataset was used to create one face template.
• Gender attribute was used to exclude comparisons between different gender subjects.
• All pre-filtered face templates from one dataset were compared with all pre-filtered templates from the other dataset.
• Each comparison score was obtained by comparing two different templates (simple one-to-one verification).
• The comparisons were fully zero-effort, meaning there was no pre-grouping by gender, age or other covariates.

Accuracy versus performance metrics

The charts below show the accuracy of all tested algorithms, with the highlighted dots corresponding to the Neurotechnology algorithm submissions.