Reliability Tests
The VeriSpeak 13.1 algorithm has been tested with voice samples taken from the XM2VTS Database, as well as with voice samples from Neurotechnology's internal dataset. Two datasets included fixed phrases pronounced by all subjects within a particular dataset, and one dataset included unique phrases for each subject.
| Voiceprint datasets used for VeriSpeak 13.1 matching engine testing | |||
|---|---|---|---|
| Experiment 1 | Experiment 2 | Experiment 3 | |
| Dataset used | XM2VTS (phrase 1) |
Neurotechnology internal dataset 1 | Neurotechnology internal dataset 2 |
| Fixed/unique phrase | Fixed | Fixed | Unique |
| Total voice samples in the dataset | 2360 | 309 | 305 |
| Subjects in the dataset | 295 | 42 | 42 |
| Recording sessions per subject | 8 | 1 - 10 | 1 - 10 |
| Average voice sample length (seconds) | 7.112 | 4.975 | 6.214 |
Three tests were performed during each experiment:
- Test 1 used text-dependent voiceprint template matching algorithm. VeriSpeak 13.1 algorithm reliability in this test is shown as red curves on the ROC charts.
- Test 2 used text-independent voiceprint template matching algorithm. VeriSpeak 13.1 algorithm reliability in this test is shown as blue curves on the ROC charts.
- Test 3 used fused results from the text-dependent and text-independent voiceprint template matching algorithms. VeriSpeak 13.1 algorithm reliability in this test is shown as green curves on the ROC charts.
Experiment 1
Click to zoom
Click to zoom
Experiment 2
Click to zoom
Click to zoom
Experiment 3
Click to zoom
Click to zoom
Receiver operation characteristic (ROC) curves are usually used to demonstrate the recognition quality of an algorithm. ROC curves show the dependence of false rejection rate (FRR) on the false acceptance rate (FAR). Charts with ROC curves for each of the experiments are available above.
| VeriSpeak 13.1 matching engine reliability tests | |||||||||
|---|---|---|---|---|---|---|---|---|---|
| Exp. 1 | Exp. 2 | Exp. 3 | |||||||
| Test 1 | Test 2 | Test 3 | Test 1 | Test 2 | Test 3 | Test 1 | Test 2 | Test 3 | |
| Average template size (bytes) | 2796 | 2796 | 2796 | 4051 | 4051 | 4051 | 5098 | 5098 | 5098 |
| EER | 3.4640 % | 0.1074 % | 0.1254 % | 1.1750 % | 0.3569 % | 0.2528 % | 0.1720 % | 0.3147 % | 0.0928 % |
| FRR at 0.1 % FAR | 18.5500 % | 0.1575 % | 0.1696 % | 3.2000 % | 0.8349 % | 0.2783 % | 0.2854 % | 0.8563 % | 0.0952 % |
| FRR at 0.01 % FAR | 32.9900 % | 0.7634 % | 0.6301 % | 6.9110 % | 3.0610 % | 1.2060 % | 0.3806 % | 3.2350 % | 1.3320 % |
| FRR at 0.001 % FAR | 47.2000 % | 2.0840 % | 1.4660 % | 8.3490 % | 6.2150 % | 3.0610 % | 0.4757 % | 5.8040 % | 3.4250 % |
