Perceptual and Motor Skills, 1990, 70, 232-234.
O Perceptual
and Motor Skills 1990
SPEECH CLARITY/INTELLIGIBILITY TEST-RETEST RELIABILITY O F MAGNITUDE-ESTIMATION SCALING ' DONALD FUCCI AND LEE ELLIS Ohio Universi~,Athens, Ohio LINDA PETROSINO Bowling Green State University, Bowling Green, Ohio Summary.-The
reliability of magnitude-estimation scaling as a measure of over-
all clarity of speech was investigated. 40 subjects (M age = 19 yr.) provided magnitude-estimation responses for nine audiotaped versions of a nonsense sentence varying systematically in number of correct consonant phonemes. There was no significant difference in the magnitude-estimation responses of the subjects during two test sessions separated by one week. Analysis suggested that magnitude-estimation scaling is a reliable measure of speech clarity/intelligibilityYThis finding is discussed in relation to speech samples varying in aspects other than number of consonant phonemes correct and possible further clinical research applications.
Procedures used for measuring the over-all clarity or intelligibility of speech have included identification methods and scahng methods. The two most common scaling methods used for this purpose are direct magnitude estimation and interval scaling. The appropriateness of direct magnitude-estimation and interval-scaling procedures for assessing speech intehgibllity has recently been investigated by Schiavetti, Metz, and Sitler (1981). These researchers determined that magnitude-estimation scaling is preferred to interval scaling because speech intelligibility is representative of a prothetic (additive) continuum. While magnitude estimation scaling appears to be an appropriate measure of clarity/intelligibihty of speech (Schavetti, Metz, & Sitler, 1781), little information is available regarding the reliability of magnitude-estimation scaling for this purpose (Metz, Samar, Schiavetti, Sitler, & Whitehead, 1985). The present study was designed to assess the reliabihty of magnitude-estimation scaling as a measure of the over-all clarity of speech samples in which the number of correctly produced English consonant phonemes were systematically varied. -
-
METHOD Subjects.-Forty subjects were randomly selected from a group of 125 students enrolled in an introductory course in speech and hearing sciences at Ohio University. The subjects (20 men and 20 women) ranged in age from 17 to 23 yr. (M age = 19 yr.). All subjects passed a pure-tone audiometric screening test at an intensity level of 25 dB hearing threshold level (HTL). They 'Request reprints from Dr. Donald Fucci, School of Hearing and Speech Sciences, Ohio University, Athens, OH 45701.
SPEECH
CLARITY/INTELLIGIBILITY:TESTRETEST
233
also showed normal speech and language skills and reported no known medical or physical conditions that could interfere with test results. Stimulus marerial.-To be certain of the differences in the stimuli to which the listeners were responding, only a single parameter of the stimuli was varied. The number of consonant phonemes produced correctly was chosen as the variable to manipulate because, of the many variables that may influence intelligibility, it seemed to be a relatively simple one to vary systematically. Also, research suggested that there is a direct relationship between number of segmental errors and intelligibility, at least for hearing impaired speakers (Osberger & Hesketh, 1988, Smith, 1975). Control of other variables, such as sentence meaning, length of stimuli, and prosodic features, was accomplished by using a single nonsense sentence described by Fairbanks (1960). A major advantage of this particular nonsense sentence is that it contains all of the consonant phonemes of English, each occurring only once. Nine audiotaped stimuli were prepared by having an adult male speaker record this nonsense sentence nine times, maintaining the same prosodic features for each utterance of the sentence but varying the number of correct phonemes produced from 0 to 24 (i.e., 0, 3, 6, 7 , 12, 15, 18, 21, 24). All "incorrect consonants" consisted of substitution errors involving either a common rnisarticuIation or an error similar to the target sound in terms of number of shared distinctive features. These erron were assigned randomly to the different stimulus sentences. A Wallensak 3M Model 2590 cassette recorder with an Audio Technica Pro-1 microphone was used to record the stimuli. With the microphone positioned approximately 12 in. from his lips, the speaker recited each stimulus sentence at an intensity level of approximately 70 dB as measured by a B & K Sound Level Meter. Procedzire.-Subjects attended two experimental sessions scheduled one week apart. The procedures during both sessions were identical. Subjects were seated at individual Listening booths at the Ohio University Language Laboratory, and each subject wore a set of TDH-49 headphones. Prior to the actual testing, the volume controls in each booth were set at the same intensity level (60 dB SPL), and all subjects reported able to hear a nontest sentence played as a system check. The same volume setting was maintained throughout all stimulus presentations. The subjects received the following instructions, which were provided on a typed handout and read aloud by the experimenter. You'll be hearing a nonsense sentence. It's made up of English words but they are not in any meanin ful order. Sometimes the sentence is clear and other times it's not so clear. After heating eack sentence, write down a number that goes with how clear the sentence sounds to you. You can use any numbers that you wish-whole numbers, decimals, or fractions. Give lower numbers for c l e a r sentences; give higher numbers for less clear sentences. To minimize possible biases in the data, no reference standard was employed (Hellman & Zwislocki, 1963). During each of the two sessions, the nine stimulus tapes were presented once in randomized order.
RESULTSAND DISCUSSION Subjects' mean magnitude estimation responses for Sessions 1 and 2 were very similar. For Session 1, the mean responses ranged from 5.9 (SD = 10.4) to 13.1 (SD = 20.7). For Session 2, the mean responses ranged from 3.7 (SD = 7.6) to 13.3 (SD = 18.6). A repeated-measures analysis of variance (Bruning & Kintz, 1987) indicated no significant difference between the magnitude-estimation scaling responses provided by the subjects in the two sessions (F,,,,= .02, p > .89). This result suggests that magnitude-estimation scaling (without a reference standard) was a reliable method for measur-
234
DONALD FUCCI, ET A L .
ing the clarity of audiotaped speech samples which varied systematically in the number of consonant phonemes produced correctly. Given its construct validity (Schiavetti, Metz, & Sitler, 1981) and test-retest reliability for measuring speech samples differing in the number of consonant phonemes produced correctly, magnitude-estimation scaling could be a useful measure of the intelligibility of speech samples varying in other aspects. Parameters such as type of error, duration, frequency, and intensity of the stimuli might be changed, separately and in combination, to estimate their relative effects on over-all clarity of speech. Applications of magnitude-estimation scaling with actual clinical populations might also be explored. For example, this technique could be used to determine whether clinicians vary significantly as a group when using magnitude-estimation scaling vs other measures to assess the over-all clarity/intelhgibility of disordered speakers. Magnitude estimation could also be used to explore possible clarity/intelligibility measures made by untrained listeners such as parents or spouses of individuals with communication problems. Clinicians and/or untrained listeners might vary systematically in their estimations of the clarity of speech of persons with different types and severities of communication disorders. I t is possible that magnitude-estimation scaling could become a reliable and valid measure of intelligibility when used as a way to measure clients' change associated with treatment over time. REFERENCES BRWG, J. L., & KINTZ, B. L. (1987) Computational handbook of statistics. (3rd ed.) Glenview, IL: Scott, Foresman. FAIRBANKS, G. (1960) Voice and articrrlation drillbook. (2nd ed.) Cambridge, MA: Harper & Row. HELLMAN, R., & ZWISLOCKI,. (1963) Monaural loudness function at 1000 cps and interaural summation. Journal olthe Acoustical Society of America, 35, 856-865. METZ, D., SAMAR,V. J., SCHIAVETTI,N., SITLER,R., & WHITEHEAD,R. (1985) Acoustic dimensions of hearing impaired speakers' intelligibihty. Journal of Speech and Hearing Research, 28, 345-355. OSBERGER, M., & HESKETH, L. (1988) Speech and language disorders related to hearing impairments. In N. Lass (Ed.), Handbook of speech-language pathology and audiology. Philadelphia, PA: Decker. SCHIAVETI~, N., METZ,D., & SITLER,R. (1981) Construct validity of direct magnitude estimation and interval scaling of speech intelligibili evidence from a study of the hearing impaired. Journal of Speech and Hearing Researc?: 24, 441-445. S M I ~ - C. I , (1975) Residual hearing and speech production in deaf children. Journal of Speech and Hearing Research, 18, 795-811.
Accepted February 2, I990
O Perceptual
and Motor Skills 1990
SPEECH CLARITY/INTELLIGIBILITY TEST-RETEST RELIABILITY O F MAGNITUDE-ESTIMATION SCALING ' DONALD FUCCI AND LEE ELLIS Ohio Universi~,Athens, Ohio LINDA PETROSINO Bowling Green State University, Bowling Green, Ohio Summary.-The
reliability of magnitude-estimation scaling as a measure of over-
all clarity of speech was investigated. 40 subjects (M age = 19 yr.) provided magnitude-estimation responses for nine audiotaped versions of a nonsense sentence varying systematically in number of correct consonant phonemes. There was no significant difference in the magnitude-estimation responses of the subjects during two test sessions separated by one week. Analysis suggested that magnitude-estimation scaling is a reliable measure of speech clarity/intelligibilityYThis finding is discussed in relation to speech samples varying in aspects other than number of consonant phonemes correct and possible further clinical research applications.
Procedures used for measuring the over-all clarity or intelligibility of speech have included identification methods and scahng methods. The two most common scaling methods used for this purpose are direct magnitude estimation and interval scaling. The appropriateness of direct magnitude-estimation and interval-scaling procedures for assessing speech intehgibllity has recently been investigated by Schiavetti, Metz, and Sitler (1981). These researchers determined that magnitude-estimation scaling is preferred to interval scaling because speech intelligibility is representative of a prothetic (additive) continuum. While magnitude estimation scaling appears to be an appropriate measure of clarity/intelligibihty of speech (Schavetti, Metz, & Sitler, 1781), little information is available regarding the reliability of magnitude-estimation scaling for this purpose (Metz, Samar, Schiavetti, Sitler, & Whitehead, 1985). The present study was designed to assess the reliabihty of magnitude-estimation scaling as a measure of the over-all clarity of speech samples in which the number of correctly produced English consonant phonemes were systematically varied. -
-
METHOD Subjects.-Forty subjects were randomly selected from a group of 125 students enrolled in an introductory course in speech and hearing sciences at Ohio University. The subjects (20 men and 20 women) ranged in age from 17 to 23 yr. (M age = 19 yr.). All subjects passed a pure-tone audiometric screening test at an intensity level of 25 dB hearing threshold level (HTL). They 'Request reprints from Dr. Donald Fucci, School of Hearing and Speech Sciences, Ohio University, Athens, OH 45701.
SPEECH
CLARITY/INTELLIGIBILITY:TESTRETEST
233
also showed normal speech and language skills and reported no known medical or physical conditions that could interfere with test results. Stimulus marerial.-To be certain of the differences in the stimuli to which the listeners were responding, only a single parameter of the stimuli was varied. The number of consonant phonemes produced correctly was chosen as the variable to manipulate because, of the many variables that may influence intelligibility, it seemed to be a relatively simple one to vary systematically. Also, research suggested that there is a direct relationship between number of segmental errors and intelligibility, at least for hearing impaired speakers (Osberger & Hesketh, 1988, Smith, 1975). Control of other variables, such as sentence meaning, length of stimuli, and prosodic features, was accomplished by using a single nonsense sentence described by Fairbanks (1960). A major advantage of this particular nonsense sentence is that it contains all of the consonant phonemes of English, each occurring only once. Nine audiotaped stimuli were prepared by having an adult male speaker record this nonsense sentence nine times, maintaining the same prosodic features for each utterance of the sentence but varying the number of correct phonemes produced from 0 to 24 (i.e., 0, 3, 6, 7 , 12, 15, 18, 21, 24). All "incorrect consonants" consisted of substitution errors involving either a common rnisarticuIation or an error similar to the target sound in terms of number of shared distinctive features. These erron were assigned randomly to the different stimulus sentences. A Wallensak 3M Model 2590 cassette recorder with an Audio Technica Pro-1 microphone was used to record the stimuli. With the microphone positioned approximately 12 in. from his lips, the speaker recited each stimulus sentence at an intensity level of approximately 70 dB as measured by a B & K Sound Level Meter. Procedzire.-Subjects attended two experimental sessions scheduled one week apart. The procedures during both sessions were identical. Subjects were seated at individual Listening booths at the Ohio University Language Laboratory, and each subject wore a set of TDH-49 headphones. Prior to the actual testing, the volume controls in each booth were set at the same intensity level (60 dB SPL), and all subjects reported able to hear a nontest sentence played as a system check. The same volume setting was maintained throughout all stimulus presentations. The subjects received the following instructions, which were provided on a typed handout and read aloud by the experimenter. You'll be hearing a nonsense sentence. It's made up of English words but they are not in any meanin ful order. Sometimes the sentence is clear and other times it's not so clear. After heating eack sentence, write down a number that goes with how clear the sentence sounds to you. You can use any numbers that you wish-whole numbers, decimals, or fractions. Give lower numbers for c l e a r sentences; give higher numbers for less clear sentences. To minimize possible biases in the data, no reference standard was employed (Hellman & Zwislocki, 1963). During each of the two sessions, the nine stimulus tapes were presented once in randomized order.
RESULTSAND DISCUSSION Subjects' mean magnitude estimation responses for Sessions 1 and 2 were very similar. For Session 1, the mean responses ranged from 5.9 (SD = 10.4) to 13.1 (SD = 20.7). For Session 2, the mean responses ranged from 3.7 (SD = 7.6) to 13.3 (SD = 18.6). A repeated-measures analysis of variance (Bruning & Kintz, 1987) indicated no significant difference between the magnitude-estimation scaling responses provided by the subjects in the two sessions (F,,,,= .02, p > .89). This result suggests that magnitude-estimation scaling (without a reference standard) was a reliable method for measur-
234
DONALD FUCCI, ET A L .
ing the clarity of audiotaped speech samples which varied systematically in the number of consonant phonemes produced correctly. Given its construct validity (Schiavetti, Metz, & Sitler, 1981) and test-retest reliability for measuring speech samples differing in the number of consonant phonemes produced correctly, magnitude-estimation scaling could be a useful measure of the intelligibility of speech samples varying in other aspects. Parameters such as type of error, duration, frequency, and intensity of the stimuli might be changed, separately and in combination, to estimate their relative effects on over-all clarity of speech. Applications of magnitude-estimation scaling with actual clinical populations might also be explored. For example, this technique could be used to determine whether clinicians vary significantly as a group when using magnitude-estimation scaling vs other measures to assess the over-all clarity/intelhgibility of disordered speakers. Magnitude estimation could also be used to explore possible clarity/intelligibility measures made by untrained listeners such as parents or spouses of individuals with communication problems. Clinicians and/or untrained listeners might vary systematically in their estimations of the clarity of speech of persons with different types and severities of communication disorders. I t is possible that magnitude-estimation scaling could become a reliable and valid measure of intelligibility when used as a way to measure clients' change associated with treatment over time. REFERENCES BRWG, J. L., & KINTZ, B. L. (1987) Computational handbook of statistics. (3rd ed.) Glenview, IL: Scott, Foresman. FAIRBANKS, G. (1960) Voice and articrrlation drillbook. (2nd ed.) Cambridge, MA: Harper & Row. HELLMAN, R., & ZWISLOCKI,. (1963) Monaural loudness function at 1000 cps and interaural summation. Journal olthe Acoustical Society of America, 35, 856-865. METZ, D., SAMAR,V. J., SCHIAVETTI,N., SITLER,R., & WHITEHEAD,R. (1985) Acoustic dimensions of hearing impaired speakers' intelligibihty. Journal of Speech and Hearing Research, 28, 345-355. OSBERGER, M., & HESKETH, L. (1988) Speech and language disorders related to hearing impairments. In N. Lass (Ed.), Handbook of speech-language pathology and audiology. Philadelphia, PA: Decker. SCHIAVETI~, N., METZ,D., & SITLER,R. (1981) Construct validity of direct magnitude estimation and interval scaling of speech intelligibili evidence from a study of the hearing impaired. Journal of Speech and Hearing Researc?: 24, 441-445. S M I ~ - C. I , (1975) Residual hearing and speech production in deaf children. Journal of Speech and Hearing Research, 18, 795-811.
Accepted February 2, I990
