Speaker Identification System Test and Results
1.2
2006/12/19 14:00:35 US/Central
2006/12/20 11:53:53.748 US/Central
Chris
Pasich
cpasich@rice.edu
Chris
Pasich
cpasich@rice.edu
An overview of the testing methods and the results of the tests.
Identity Checking Results
For testing purposes, we had five sounds that
the system would test – three vowel sounds, and two words. Each
member of the group tested a each sound 5 times. The possible
results for a test are as follows: match (M), meaning the system
identified the speaker correctly; incorrect match (IM), meaning the
system identified the speaker incorrectly; or no match (NM) meaning
the speaker did not find the correct speaker in the
database.
Identity Checking Test Results
“Ah”
“Oh”
“Ay”
“Avocado”
“Diablo”
Speaker
M
IM
NM
M
IM
NM
M
IM
NM
M
IM
NM
M
IM
NM
Damen Hattori
4
1
0
4
1
0
3
2
0
2
3
0
4
1
0
Josh Long
3
2
0
4
1
0
2
3
0
5
0
0
3
2
0
Matt McDonell
1
4
0
5
0
0
3
1
1
4
1
0
5
0
0
Chris Pasich
4
1
0
2
1
2
2
1
2
3
1
1
2
3
0
Overall
12
8
0
15
3
2
10
7
3
14
5
1
14
6
0
Overall PercentCorrect
60%
75%
50%
70%
70%
Overall, the system identified speakers
correctly 67% of the time. On an individual basis, Matt McDonell
was recognized most often (72%), Damen Hattori and Josh Long were
recognized correctly equally as often (68%) and Chris Pasich was
recognized correctly with the least frequency (60%). Overall,
however, all speakers were identified at a fairly good rate, given
the complexity of the system.
Vowel Checking Results
In addition to testing whether a speaker was
identified correctly, we also tested to see if the system correctly
identified vowel sounds. The vowel sounds were either found or not
found, and were never incorrectly identified. The overall results
are listed in the Vowel Checking Results below.
Vowel Checking Test Results
“Ah”
“Oh”
“Ay”
“Avocado”
“Diablo”
Vowel Found
20
18
17
71
49
Vowel Not Found
0
2
3
9
11
% Vowels Found
100%
90%
85%
88.75%
81.7%
Overall, the system correctly identified 87.5%
of all vowels correctly, an extremely high rate for a vowel
checking system. As the word became more complicated, the vowels
were not found as frequently. This is a result of the added
syllables and the emphasis on the consonants in the words.
Results Overview
Overall, our results were acceptable for a
system of this much complexity. A system that correctly identifies
the speaker with 67% accuracy is not good for security purposes,
but with fine tuning and more time, the accuracy could easily
increase. One of the more important results from our testing is
that, as the complexity of a spoken word increased, the accuracy of
the system also slightly increased. There is much more room for
error with longer words than with single-syllable vowels, and this
is reflected in the overall increase in accuracy.