Summary: An overview of the testing methods and the results of the tests.
For testing purposes, we had five sounds that the system would test – three vowel sounds, and two words. Each member of the group tested a each sound 5 times. The possible results for a test are as follows: match (M), meaning the system identified the speaker correctly; incorrect match (IM), meaning the system identified the speaker incorrectly; or no match (NM) meaning the speaker did not find the correct speaker in the database.
| “Ah” | “Oh” | “Ay” | “Avocado” | “Diablo” | |||||||||||
| Speaker | M | IM | NM | M | IM | NM | M | IM | NM | M | IM | NM | M | IM | NM |
| Damen Hattori | 4 | 1 | 0 | 4 | 1 | 0 | 3 | 2 | 0 | 2 | 3 | 0 | 4 | 1 | 0 |
| Josh Long | 3 | 2 | 0 | 4 | 1 | 0 | 2 | 3 | 0 | 5 | 0 | 0 | 3 | 2 | 0 |
| Matt McDonell | 1 | 4 | 0 | 5 | 0 | 0 | 3 | 1 | 1 | 4 | 1 | 0 | 5 | 0 | 0 |
| Chris Pasich | 4 | 1 | 0 | 2 | 1 | 2 | 2 | 1 | 2 | 3 | 1 | 1 | 2 | 3 | 0 |
| Overall | 12 | 8 | 0 | 15 | 3 | 2 | 10 | 7 | 3 | 14 | 5 | 1 | 14 | 6 | 0 |
| Overall PercentCorrect | 60% | 75% | 50% | 70% | 70% | ||||||||||
Overall, the system identified speakers correctly 67% of the time. On an individual basis, Matt McDonell was recognized most often (72%), Damen Hattori and Josh Long were recognized correctly equally as often (68%) and Chris Pasich was recognized correctly with the least frequency (60%). Overall, however, all speakers were identified at a fairly good rate, given the complexity of the system.
In addition to testing whether a speaker was identified correctly, we also tested to see if the system correctly identified vowel sounds. The vowel sounds were either found or not found, and were never incorrectly identified. The overall results are listed in the Vowel Checking Results below.
| “Ah” | “Oh” | “Ay” | “Avocado” | “Diablo” | |
| Vowel Found | 20 | 18 | 17 | 71 | 49 |
| Vowel Not Found | 0 | 2 | 3 | 9 | 11 |
| % Vowels Found | 100% | 90% | 85% | 88.75% | 81.7% |
Overall, the system correctly identified 87.5% of all vowels correctly, an extremely high rate for a vowel checking system. As the word became more complicated, the vowels were not found as frequently. This is a result of the added syllables and the emphasis on the consonants in the words.
Overall, our results were acceptable for a system of this much complexity. A system that correctly identifies the speaker with 67% accuracy is not good for security purposes, but with fine tuning and more time, the accuracy could easily increase. One of the more important results from our testing is that, as the complexity of a spoken word increased, the accuracy of the system also slightly increased. There is much more room for error with longer words than with single-syllable vowels, and this is reflected in the overall increase in accuracy.