JSLHR
Research Note
Improving Child Compliance on a Computer-Administered Nonword Repetition Task Kamila Polišenskáa and Svetlana Kapalkováb
Purpose: A range of nonword repetition (NWR) tasks are used in research and clinical applications, but compliance rates among young children remain low. Live presentation is usually used to improve compliance rates, but this lacks the consistency of recorded stimuli. In this study, the authors examined whether a novel delivery of NWR stimuli based on recorded material could provide improved compliance rates in young children, thereby reducing research bias. Method: The novel NWR task with 26 recorded items was administered to 391 typically developing children ages 2–6 years. The children were presented with a story that they could influence by repeating “magic” words. The task was administered via computer with animation.
Results: From the 384 children who completed the task, the authors found a noncompliance rate related to age. In line with previous research, no effect of demographic factors was found, but there was a significant main effect of age, syllable length, and phonological complexity on repetition accuracy. Test– retest and interrater scoring showed high levels of reliability. Conclusion: The task described in this study offers an objective delivery of recorded stimuli that engages young children and provides high compliance rates. The task is inexpensive, requires minimal training, and can be adapted to other languages.
S
children, particularly under age 3 years, and tests that can be administered quickly are advantageous due to shorter attention spans. In this age group, direct assessments frequently have high noncompliance rates, so parental reports are often utilized as an alternative source of information about language abilities (Fenson et al., 1993; Sachse & Von Suchodoletz, 2008). The main disadvantage of indirect parental reports and checklists is that parents might underestimate or overestimate their children’s skills. Semistructured play interviews have been suggested as an alternative to formal assessment or parental reports (Brassard & Boehm, 2011; Lahey, 1988). Although assessment through play can be informative, analysis and interpretation can be timeconsuming and require the supervision of an experienced clinician. The aim of this study was to develop a NWR task for children between 2 and 6 years of age that can be easily administered, may be analyzed with minimal training, and can yield high levels of compliance. The relationship between language skills and NWR performance in young children has been shown in a number of studies. Hoff, Core, and Bridges (2008) reported significant correlations between vocabulary percentiles and NWR accuracy in two studies with children ages 1;8 to 2;0 (years; months), r(13) = .72, p < .001 and r(19) = .53, p < .01, respectively, and Stokes and Klee (2009) showed that NWR
uccessful assessments of children’s psycholinguistic processing and learning skills minimize the influence of prior experience and/or environmental factors on the resulting scores. Nonword repetition (NWR) tasks are seen as being less affected by environmental factors than traditional language tests (Campbell, Dollaghan, Needleman, & Janosky, 1997; Engel, Santos, & Gathercole, 2008). NWR tasks have also proven to be effective clinical markers for language impairment in a number of languages (English: see Coady & Evans, 2008 for review; Italian: Bortolini et al., 2006; Spanish: Girbau & Schwartz, 2007; Icelandic: Thordardottir, 2008; Slovak: Kapalková, Polišenská, & Vicenová, 2013) and have shown promise as a clinical tool (Archibald, 2008). Another important aspect of assessment is the opportunity to provide early identification of language deficits (e.g., Bercow, 2008; Rossetti, 2001). However, it is often challenging to assess the language abilities of younger
a
University of Manchester, United Kingdom Comenius University, Bratislava, Slovakia
b
Correspondence to Kamila Polišenská: [email protected] Editor: Rhea Paul Associate Editor: Stephanie Stokes Received January 16, 2013 Revision received May 17, 2013 Accepted August 19, 2013 DOI: 10.1044/1092-4388(2013/13-0014)
1060
Key Words: children, assessment, screening, phonology
Disclosure: The authors have declared that no competing interests existed at the time of publication.
Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014 • A American Speech-Language-Hearing Association
in English-speaking 2-year-olds distinguished late talkers from typically developing (TD) children, possibly due to the close links between phonological processing and lexical development. Similarly, NWR performance in Englishspeaking 2- to 3-year-olds was related to their morphosyntactic skills at ages 4 to 5 years, r(145) = .32, p < .001 (Chiat & Roy, 2008). These findings suggest that NWR performance in 2-year-old children is informative and can predict broader language skills, but potentially inconsistent delivery resulting from the use of nonrecorded stimuli and high noncompliance rates have remained a concern. Hoff et al.’s first NWR experiment with 16 TD children ages 1;9 to 2;0 had an impressive noncompliance rate of 6%, but the second attempt with the same age group (1;8–2;0) but different set of stimuli yielded a noncompliance rate of 22%, with six out of 27 children failing to complete the task. Stokes and Klee (2009) reported noncompliance rates of 20% and 27% in two versions of their Test of Early Nonword Repetition with a large sample of 2-year-old children, and an even higher noncompliance rate was reported in a sample of 2-year-old TD Dutch-speaking children tested by Zamuner (2009), with 37 out of 101 children (37%) not finishing the NWR task. Studies with samples including slightly older children reported better cooperation figures. Roy and Chiat (2004) carried out a study with 2- to 4-year-olds and reported a loss of 11% children due to noncooperation. Seeff-Gabriel, Chiat, and Roy (2008) tested children ages 2;0–5;11 and reported a refusal rate of 5.3% on their NWR and word repetition task administered live. The majority of children (90%) who refused to cooperate were children under 3. It has been suggested that noncompliance might be related to deficit (Hoff et al., 2008). Stokes and Klee (2009) showed that children who failed to complete their NWR task scored significantly lower on the MacArthur-Bates Communicative Development Inventory (CDI) and also on receptive and expressive vocabulary tasks than the children who completed the NWR task. However, it is unlikely that the proportion of children with a deficit would be as high as suggested by noncompliance rates of over 20% as reported in some of the studies discussed above, and it remains unclear which nonresponding children have a deficit and which nonrespondents are just unwilling to cooperate. We are not examining the reasons behind noncompliance but are attempting to increase the number of children who complete the task in order to reduce nonrespondent bias. In addition, studies on 2-year-old children have so far all opted to use live presentation rather than recorded stimuli in the belief that live presentation will lead to increased engagement and therefore higher completion rates. Live presentation is more flexible and may well assist in increasing engagement through greater opportunity for interaction, but it does mean that the delivery will be less consistent, and there is a greater likelihood of production errors by the tester. Our main aim was to develop a methodology for language assessments that could utilize the consistency of recorded nonword stimuli to minimize research bias while still effectively engaging children as young as 2 years. We also wanted to establish validity and reliability of this newly developed task.
An additional aim was to examine whether our NWR task was influenced by environmental factors such as socioeconomic status (SES). Campbell et al. (1997) tested school-aged boys from minority and majority backgrounds on a “knowledge dependent” measure that relied heavily on vocabulary and also “processing dependent” measures such as a Nonword Repetition Task (NRT; Campbell, Needleman, Riess, & Tobin, 1995). The study found that boys from minority backgrounds scored significantly lower than boys from majority backgrounds on the knowledge dependent measure but did not differ on the NRT, leading to the conclusion that processing dependent measures are not biased against minority speakers. More recently, a study by Engel et al. (2008) investigated the effect of Brazilian children’s SES on performance on vocabulary and NWR tasks. Children from the lower SES group scored significantly lower than children from the higher SES group on receptive and expressive vocabulary tests, but there was no significant difference on NWR scores. Reynolds and Fish (2010) reported similar findings for low-SES rural Appalachian children who scored lower compared to population scores on traditional language tests, but their mean nonword score was almost identical to the nonminority group from Campbell et al. (1997). These findings collectively support the idea that NWR tasks are less influenced by prior knowledge than traditional language tests. However, this conclusion has been called into question by a recent study by Roy and Chiat (2013) who found that English-speaking children from a low SES background scored significantly lower than children from a high SES background on both repetition tasks and more traditional assessments. As a result, the present study also investigated the effect of SES on NWR performance.
Method Participants The study was conducted according to the Slovakian ethical guidelines for research with human subjects. Parents of children in 15 nurseries in Slovakia were provided with an information sheet and consent form. Four hundred and two parents consented for their child to take part in the study. Consenting parents were also asked to fill in a questionnaire about parental education, the child’s date of birth, and any known developmental disorders, hearing problems, or concerns about their child’s development. Information about parental education was not always completed, and therefore this part of the data is sometimes missing (see the Results section for more detail). Only children who spoke Slovak as a first language and had no known developmental disorders were included in the analyses. This meant that 11 children were excluded: three children had a clinical diagnosis of a developmental disorder, six children did not speak Slovak as their first language, and information about two children’s date of birth was incomplete. Therefore, the final sample consisted of 391 children. From this sample, seven children failed to complete the task. Nurseries in both rural and urban areas were contacted. The final sample
Polišenská & Kapalková: Child Compliance on a Nonword Repetition Task
1061
consisted of 61 children (16%) from nurseries in a rural area and 323 children (84%) from nurseries in urban areas. A majority of the children (77%) attended state nurseries and 23% attended private nurseries. Table 1 shows the participants’ characteristics and the noncompliance rate for each age group and the total sample. A child was classified as noncompliant if she or he refused to attempt the task at all. Once the child started the test, nonresponses were very rare, and all children who started the test completed it.
presented on a laptop computer in a quiet room during a session that lasted about 5 min. The NWR task was embedded in a story:
Materials
The story was told in simple sentences with no sentence exceeding five words in the Slovak version. The simple structure and short sentences were chosen to ensure that even the youngest children could comprehend the story and instructions. There were two practice items and 26 nonword items in the full test. The nonwords were presented in a fixed random order. The sound was presented over external speakers connected to the laptop. All of the items were recorded in a professional sound studio in Bratislava and were edited using Audacity sound software. The sound files were embedded in a PowerPoint file and accompanied by the pictures presented in Figure 1. The experimenter controlled when each nonword was played. After a child repeated or attempted to repeat the item, a bead appeared on the necklace. Each item was played a maximum of two times. Every bead appeared with an animated effect and stayed on the screen, allowing the children to see their progress. In the end, all 26 beads appeared, and Hugo and Nina were shown smiling with the completed necklace for their Mom. All of the children’s responses were scored online and also recorded on a digital audio recorder. At the end of the session, the investigator verbally praised each child (rather than, for example, giving stickers); the child was shown a slide with the completed necklace they had made. The whole experiment is presented as a video in the supplemental materials. An editable PowerPoint presentation is available on request from the first author.
The task consisted of 26 nonwords with varying syllable length and phonological complexity (for a full set of items, see the Appendix). Some items were taken from a task developed by Kapalková et al. (2013) and new items were also created to control for phonological complexity. Items with CV, CVC, VC structures were considered phonologically simple, whereas CCV, CCVC, CVCC items were considered phonologically complex. There were four monosyllabic items: two phonologically simple without consonant clusters and two with a single consonant cluster either in the onset or coda. Similarly, the eight disyllabic items were divided between four phonologically simple and four complex stimuli. The same pattern also applied to the eight trisyllabic items: Four had a simple syllable structure and four a complex syllable structure. Four-syllable items were represented by one simple and one phonologically complex item, and this was also the case for five-syllable items. The total of 24 items could be categorized by length: 12 short items (1- and 2-syllable items) versus 12 longer items (3-, 4-, and 5-syllable items). The task also included two items with the consonants /l/ and /r/ as a syllable nucleus as the Slovak language allows syllabic /r/ and /l/ to form the nucleus of a syllable; such words are common in Slovak.
Procedure Each child was tested individually in a nursery by one of six qualified speech and language therapists who were all native speakers of Slovak. The tester showed the child the laptop and told him or her that it had games on it. The tester asked the child if he or she wanted to play a game on the computer, and if the child agreed, he or she was shown the NWR task (note that noncompliance occurred when children refused to play a game on the computer). The task was
Hugo and Nina made a necklace for their Mom. They used colorful beads. But the necklace broke. And the beads all fell on the floor. Hugo and Nina were very sad. But we can help them. With magic words, we can make a new necklace. When you repeat the magic word, a bead will appear on the necklace. Now let’s try it.
Scoring—Whole Items Each child’s responses were scored online during the experiment by the tester. All of the sessions were recorded, and a random subset of children from the recordings were also scored offline to determine reliability (see the Reliability section, below). Both the online and offline scoring used the same criteria: A correct repetition of a whole item earned
Table 1. Participants’ characteristics and noncompliance rate according to age groups. Participants
2-year-olds
3-year-olds
4-year-olds
5-year-olds
6-year-olds
Total
Recruited sample Noncompliance rate (%) Compliance rate (%) Final sample Mage in months (SD) Gender (F/M)
n = 28 14.3 85.7 n = 24 30.13 (3.51) 14/10
n = 63 4.8 95.2 n = 60 42.13 (2.76) 32/28
n = 113 0 100 n = 113 53.69 (3.79) 65/48
n = 115 0 100 n = 115 65.07 (3.52) 55/60
n = 72 0 100 n = 72 75.68 (3.51) 40/32
n = 391 1.79 98.21 n = 384 57.94 (13.4) 206/178
Note. F = female; M = male.
1062 Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014
Figure 1. Pictures accompanying the story in a nonword repetition task.
1 point and an incorrect repetition earned 0 points. A correct repetition meant that all consonants and all vowels were present and were repeated in the correct order. Any changes to the original sequences of phonemes were considered incorrect, and 0 points were awarded. The maximum number of points was 26. The aim of our task was to create an instrument that could be used in clinical practice for screening purposes—therefore featuring a quick and straightforward method of scoring—that could also be adopted by speech and language therapists, assistants, and teaching assistants (see the Reliability section). As it can be assumed that teaching assistants would have a limited knowledge of developmental speech errors, we opted for a strict scoring, which did not allow for these errors. Kapalková et al. (2013) explored several methods of scoring for Slovak and wholeitem scoring appeared to be clinically most relevant. Similar to other tests used in clinical practice (e.g., the Children’s Test of Nonword Repetition, Gathercole & Baddeley, 1996; The Preschool Repetition Test, Seeff-Gabriel et al., 2008), the whole-item scoring was preferred to other types of scoring, including phoneme percentage correct. This was also supported by a study by Roy and Chiat (2004), who compared the whole-item scoring and phoneme percentage correct on a sample of English-speaking children; they concluded that the scoring methods were equal at differentiating between TD and clinical samples, but whole-item scoring was quicker.
Results An analysis of variance (ANOVA) was conducted with type of nursery (two levels: state or private), nursery location (two levels: rural or urban), and gender (two levels: girls or boys) as between-subject factors. Age was entered into the analysis as a covariate: Previous studies (e.g., Chiat & Roy, 2007; Kapalková et al., 2013) showed that repetition
performance is age related. Type of nursery was not significantly associated with children’s performance on our NWR test, F(1, 377) = 1.70, p = .193, h = .004. Nursery location was also not significantly contributing to the differences, F(1, 377) = 0.06, p = .81, h = .000. Similarly, the difference between repetition performance of girls and boys was not significant, F(1, 377) = 0.81, p = .37, h =.002. Accordingly, all three factors were excluded from subsequent analyses. We also ran another analysis to assess whether maternal education had a significant effect. As previously mentioned, information on maternal education was not available for all of the children, so only 163 children were included in the analysis. The data on maternal education were divided into three categories: (a) children of mothers who did not pass the maturitná skúška (the Slovak national exam required for entry into university), (b) children of mothers who passed the maturitná skúška, and (c) children of mothers with a university degree. There was no main effect of maternal education, F(2, 159) = 1.64, p = .198, h = .02. The only significant factor in the between-subject analyses was age, and as a result, the data were collapsed across all nonsignificant effects for further analyses, with age remaining the only between-subject variable. A mixed ANOVA with Age as a between-subject variable and Length and Phonological Complexity as withinsubject variables was carried out to investigate whether the performance on the task replicated well-established findings reported in previous studies (e.g., Chiat & Roy, 2007; Jones, Tamburelli, Watson, Gobet, & Pine, 2010). The ANOVA revealed a significant main effect of Age, F(4, 379) = 65.03, p < .001, h = .41; length, F(1, 379) = 385.28, p < .001, h = .50; and phonological complexity, F(1, 379) = 79.04, p < .001, h = .17. There were also significant two-way interactions between Length × Age, F(4, 379) = 32.51, p .05 r(18) = .27, p > .05
Polišenská & Kapalková: Child Compliance on a Nonword Repetition Task
1065
the same or lower noncompliance rates as previous studies in children as young as 2 years. Archibald (2008, p. 25) suggested that “clinicians may unwittingly influence the nonword repetition performance of their clients by hyperarticulating more difficult or longer nonwords, or when clients are doing poorly.” The recorded stimuli in our study meant that participants were presented with identical prosody, speech rate, and volume. Consistent presentation of NWR stimuli is particularly important: Successful repetition is wholly dependent on accurate perception of the stimuli and the unfamiliar nature of the nonlexical items may be more likely to result in production errors by the tester. Our task contained 26 items, more than any other NWR task for young children, for example, two versions of the Test of Early Nonword Repetition (Stokes & Klee, 2009), which contain 12 or 16 nonword items; the Preschool Repetition Test by Seeff-Gabriel et al. (2008), which contains 18 nonword items; and Hoff et al. (2008) with nine nonwords in Study 1 and 12 nonwords in Study 2. In spite of the higher number of items in our study, all of the children who started the task completed the repetition of all 26 items. There are a number of advantages in having a task that can successfully incorporate more items. Having more items means that more individual measurements are possible, which in turn leads to greater testing reliability as a result of the reduced likelihood of measurement error. This is particularly important when the tester employs online whole-item scoring, which is less time-consuming and therefore offers better clinical utility. Our reliability measures suggested high level of agreement between online and offline scoring, and similar findings have been reported previously (Archibald, 2008; Seeff-Gabriel et al., 2008). Furthermore, having a task that can successfully incorporate more items allows for more items per variable, and this can again increase reliability. There is also the potential to use one set of items for different age groups but without older children scoring at ceiling. Our nonwords were designed to allow for the examination of common phonological processes. This was motivated by the previous finding that there is link between language impairment and systematic phonological errors on an NWR task (Kapalková et al., 2013). Having more items allows for a greater number of phonological processes to be examined and has the added advantage of allowing for the same process to be examined on multiple occasions. Because the task is recorded, a clinician can address the consistency of the phonological processes in greater detail rather than having to rely on online judgments. This is particularly useful in Slovak and other languages where no other expressive phonological assessments are available. Overall, the noncompliance rate was 1.79% for children ages 2 to 6 years. This reflects the simple and straightforward nature of the task, with its clear aim and easy to follow rules. The way that the children willingly participated without a direct reward such as stickers reinforces how the task appealed to the children. Our testers also reported that children seemed to like the fact that they could clearly see their progress: the beads on the screen showed how many nonwords had already been repeated and also how many
more items needed to be repeated in order to finish the necklace. Similar to Seeff-Gabriel et al. (2008), the noncompliance rate in our sample was age related, with the highest noncompliance rate in the youngest group. Our noncompliance rate of 14.3% was still lower than the rate reported in other studies with 2-year-olds (e.g., Stokes and Klee, 2009: 20%; Zamuner, 2009: 37%). A similar noncompliance rate (11%) was achieved by Chiat and Roy (2007) but with live presentation. Their study assessed both word and nonword repetition, and nonresponses were affected significantly by the lexical status of items. The rate of nonresponse for nonwords was actually significantly higher than the rate of nonresponse for words, suggesting that motivating children to produce nonwords is more challenging than word repetition. It should be noted that the noncompliance rate in our study reflects outright refusals, meaning that the children who refused to participate did not even begin the task. The children who failed to attempt the test also refused other activities suggested by the experimenter such as block building, playing with a toy car, or drawing. The nursery teachers also reported that these children showed less interest in language, rarely asked questions, or made comments. Although our task did engage even the youngest children in our sample, the noncompliance rate was still 14.3%. The youngest children do not cooperate for a number of different reasons such as short attention span, shyness, anxiety, and tiredness, and it is important to investigate the reasons behind children’s noncooperation. Recent work on this was reported in a study by Dohmen, Chiat, and Roy (2013), who explored the reasons for noncompliance in verbal and nonverbal imitation tasks in children ages 2 to 3. Further research could investigate the profiles of children who do not cooperate, for example by using parents’ or teachers’ questionnaires or spontaneous speech samples. The aim of our study was to decrease the refusal rate, thereby reducing the number of cases when clinicians need to interpret refusal. The task was proven to be reliable, with high levels of interrater reliability, and it was shown to produce a stable score over a short time gap after retesting children within a period of 2 weeks. In addition, the scores did not differ across children from different backgrounds or between girls or boys, confirming the finding of previous studies that NWR tasks are less influenced by environmental and demographic factors (Campbell et al., 1997). Maternal education was used in this study as the basis for the children’s SES, and it is possible that other factors (e.g., income or lack of qualifications) might lead to different results as found by Roy and Chiat (2013). Our task replicated previous findings related to the phonological properties of NWR tests in Germanic languages, such as syllable length and syllable complexity (e.g., Archibald & Gathercole, 2006; Jones et al., 2010). The reasons for the lack of the length effect, and the complexity effect in the group of 5-year-olds in particular, are unclear. This lack of difference was not related to any particular tester. The data samples for all three testers who administered the task to a group of 5-year-olds showed no significant difference. Further research is needed to clarify this finding.
1066 Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014
The NWR task used in this study is quick (approximately 5 min), easy to administer (minimal training required), and inexpensive (basic equipment needed and low cost of the test). This opens up the possibility that in addition to qualified clinicians, clinical assistants and teachers could make use of this test. Although the reliability on students’ scoring looks promising, further research is needed to assess teacheradministered reliability. In addition, further research is needed to assess online reliability of less experienced individuals such as assistants, as the current study only reported offline scoring. With this instrument, researchers have a delivery method that is easy to use, can cover multiple variables, and is suitable for 2- to 6-year-old children. In this study, we found correlations between NWR and other language assessments, and in particular, we demonstrated significant ties between grammatical skills and NWR performance and vocabulary and NWR scores, replicating earlier findings from English on significant correlations between CDI and NWR scores (e.g., Hoff et al., 2008). We were able to use a standardized assessment only in the youngest group; the issue of lack of standardized assessments has been discussed above. Upon the publication of our current results, including the video, we encourage researchers working in other languages to demonstrate concurrent validity. Another question for future research concerns the diagnostic accuracy of our NWR task. In the past, consistency was sacrificed in the hope of increasing the participation of younger participants, but our method provides both consistency of delivery and high compliance rates. Our task produces high levels of compliance while offering an objective and engaging delivery of recorded stimuli.
Acknowledgments This work was supported by contract no. APVV-0410-11 from the Slovak Research and Development Agency. The writing of this article was supported by COST Action IS0804, “Language Impairment in a Multilingual Society: Linguistic Patterns and the Road to Assessment” (http://www.bi-sli.org). We would also like to thank Zuzana Vicenová, Lenka Distlerová, Michaela Halamová, Martina Sussová, and Simona Mihaliková for their assistance with data collection, J. D. Fenton (www.edthis.com) for editing, and to all of the nursery staff members, parents, and children who participated.
References Archibald, L. M. D. (2008). The promise of nonword repetition as a clinical tool. Canadian Journal of Speech-Language Pathology and Audiology, 32, 21–28. Archibald, L. M. D., & Gathercole, S. E. (2006). Nonword repetition: A comparison of tests. Journal of Speech, Language, and Hearing Research, 49, 970–983. Bercow, J. (2008). The Bercow Report: A review of services for children and young people (0–19) with speech, language and communication needs. Nottingham, United Kingdom: DCSF Publications. Bortolini, U., Arfé, B., Caselli, C. M., Degasperi, L., Deevy, P., & Leonard, L. B. (2006). Clinical markers for specific language
impairment in Italian: the contribution of clitics and non-word repetition. International Journal of Language and Communication Disorders, 41, 695–712. Brassard, M. R., & Boehm, A. E. (2011). Preschool assessment: Principles and practices. New York, NY: Guilford Press. Campbell, T. F., Dollaghan, C., Needleman, H., & Janosky, J. (1997). Reducing bias in language assessment: Processingdependent measures. Journal of Speech, Language, and Hearing Research, 40, 519–525. Campbell, T. F., Needleman, H. L., Riess, J. A., & Tobin, M. J. (1995, June). Bone lead level and language processing performance. Poster session presented at the annual meeting of the Symposium on Research in Child Language Disorders, Madison, WI. Chiat, S., & Roy, P. (2007). The preschool repetition test: An evaluation of performance in typically developing and clinically referred children. Journal of Speech, Language, and Hearing Research, 50, 429–443. Chiat, S., & Roy, P. (2008). Early phonological and sociocognitive skills as predictors of later language and social communication outcomes. Journal of Child Psychology and Psychiatry, 49, 635–645. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. Coady, J. A., & Evans, J. L. (2008). Uses and interpretations of nonword repetition tasks in children with and without specific language impairments (SLI). International Journal of Language and Communication Disorders, 43, 1–40. Conti-Ramsden, G., Botting, N., & Faragher, B. (2001). Psycholinguistic markers for specific language impairment (SLI). Journal of Child Psychology and Psychiatry and Allied Disciplines, 42, 741–748. Dohmen, A., Chiat, S., & Roy, P. (2013). Nonverbal imitation skills in children with specific language delay. Research in Developmental Disabilities, 34, 3288–3300. Engel, P. M. J., Santos, F. H., & Gathercole, S. E. (2008). Are working memory measures free of socioeconomic influence? Journal of Speech, Language, and Hearing Research, 51, 1580–1587. Fenson, L., Dale, P., Reznick, J. S., Thal, D., Bates, E., Hartung, J. P., Pethick, S., & Reilly, J. S. (1993). The MacArthur–Bates Communicative Development Inventories: User’s guide and technical manual. Baltimore, MD: Brookes. Gathercole, S. E., & Baddeley, A. D. (1996). The Children’s Test of Nonword Repetition. London, United Kingdom: The Psychological Corporation. Girbau, D., & Schwartz, R. G. (2007). Non-word repetition in Spanish-speaking children with specific language impairment (SLI). International Journal of Language and Communication Disorders, 42, 59–75. Hanulíková, A., & Hamann, S. (2010). Slovak. Journal of the International Phonetic Association, 40, 373–378. Hoff, E., Core, C., & Bridges, K. (2008). Non-word repetition assesses phonological memory and is related to vocabulary development in 20- to 24-month-olds. Journal of Child Language, 35, 903–916. Jones, G., Tamburelli, M., Watson, S. E., Gobet, F., & Pine, J. M. (2010). Lexicality and frequency in specific language impairment: Accuracy and error data from two nonword repetition tests. Journal of Speech, Language, and Hearing Research, 53, 1642–1655. Kapalková, S., Polišenská, K., & Vicenová, Z. (2013). Non-word repetition performance in Slovak-speaking children with and without SLI: Novel scoring methods. International Journal of Language and Communication Disorders, 48, 78–89.
Polišenská & Kapalková: Child Compliance on a Nonword Repetition Task
1067
Kapalková, S., Slančová, D., Bónová, I., Kesselová, J., & Mikulajová, M. (2010). Hodnotenie komunikačných schopností v ranom veku [Evaluation of communication skills in early age]. Bratislava, Slovakia: Slovenská asociácia logopédov. Lahey, M. (1988). Language disorders and language development. New York, NY: Macmillan. Reynolds, M. E., & Fish, M. (2010). Language skills in low-SES rural Appalachian children: Kindergarten to middle childhood. Journal of Applied Developmental Psychology, 31, 238–248. Rossetti, L. M. (2001). Communication intervention: Birth to three. San Diego, CA: Singular–Thompson Learning. Roy, P., & Chiat, S. (2004). A prosodically controlled word and nonword repetition task for 2- to 4-year-olds: Evidence from typically developing children. Journal of Speech, Language, and Hearing Research, 47, 223–234. Roy, P., & Chiat, S. (2013). Teasing apart disadvantage from disorder. In C. R. Marshall (Ed.), Current issues in developmental disorders (pp. 125–150). Hove, United Kingdom: Psychology Press. Sachse, S., & Von Suchodoletz, W. (2008). Early identification of language delay by direct language assessment or parent report? Journal of Developmental and Behavioral Pediatrics, 29, 34–41. Seeff-Gabriel, B., Chiat, S., & Roy, P. (2008). The early repetition battery. Oxford, United Kingdom: Harcourt Assessment.
Stokes, S. F., & Klee, T. (2009). The diagnostic accuracy of a new Test of Early Nonword Repetition for differentiating late talking and typically developing children. Journal of Speech, Language, and Hearing Research, 52, 872–882. Stokes, S. F., Wong, A. M.-Y., Fletcher, P., & Leonard, L. B. (2006). Nonword repetition and sentence repetition as clinical markers of specific language impairment: The case of Cantonese. Journal of Speech, Language, and Hearing Research, 49, 219–236. Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use. Oxford, United Kingdom: Oxford University Press. Thordardottir, E. (2008). Language-specific effects of task demands on the manifestation of specific language impairment: A comparison of English and Icelandic. Journal of Speech, Language, and Hearing Research, 51, 922–937. van Daal, J., Verhoeven, L., van Leeuwe, J., & van Balkom, H. (2008). Working memory limitations in children with severe language impairment. Journal of Communication Disorders, 41, 85–107. Zamuner, T. S. (2009). Phonotactic probabilities at the onset of language development: Speech production and word position. Journal of Speech, Language, and Hearing Research, 52, 49–60.
Appendix Test Items Items
No cluster
Syllable structure
Cluster
Syllable structure
Practice trials
/mik/
CVC
/kra:ta/
CCV.CV
1-syllable
/sɛp/a /tʃu:t/
CVC CVC
/pisc/ /drup/a
CVCC CCVC
2-syllable
/dɔɲa/a /lumaj/a /ʒafi:t/b /i:stap/ /kr̩ɲɛ/
CV.CV CV.CVC CV.CVC VC.CVC CS.CV
/tʃ la:ta/ /plinɛt/ /zlixa/ /ɦa:kisc/
CCV.CV CCV.CVC CCV.CV CV.CVCC
3-syllable
/zieɟɪni/a /tutsifa:n/ /la:sminɔ/a /ɦɔpɛʃka/ /tʃɛvl ̩nɔ/
CV.CV.CV CV.CV.CVC CVC.CV.CV CV.CVC.CV CV.CS.CV
/spimaka/c /mu:grata/ /spɔxtaɲɛ/ /kvipɛta/
CCV.CV.CV CV.CCV.CV CCVC.CV.CV CCV.CV.CV
4-syllable
/bazɔtʃɛjɲi:k/d
CV.CV.CVC.CVC
/plɛbɔtɲina/
CCV.CVC.CV.CV
5-syllable
/ʃɔkamɔti:va/a
CV.CV.CV.CV.CV
/sputrabicɛʎkɔ/
CCV.CCV.CV.CVC.CV
a Items taken from a nonword repetition task described in Kapalková et al. (2013). bModified from /ʒafi:r/, from Kapalková et al. (2013). cModified from /spima:tʃ ka/, from Kapalková et al. (2013). dModified from /barɔtʃɛjɲi:k/, from Kapalková et al. (2013).
1068 Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014
Copyright of Journal of Speech, Language & Hearing Research is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Research Note
Improving Child Compliance on a Computer-Administered Nonword Repetition Task Kamila Polišenskáa and Svetlana Kapalkováb
Purpose: A range of nonword repetition (NWR) tasks are used in research and clinical applications, but compliance rates among young children remain low. Live presentation is usually used to improve compliance rates, but this lacks the consistency of recorded stimuli. In this study, the authors examined whether a novel delivery of NWR stimuli based on recorded material could provide improved compliance rates in young children, thereby reducing research bias. Method: The novel NWR task with 26 recorded items was administered to 391 typically developing children ages 2–6 years. The children were presented with a story that they could influence by repeating “magic” words. The task was administered via computer with animation.
Results: From the 384 children who completed the task, the authors found a noncompliance rate related to age. In line with previous research, no effect of demographic factors was found, but there was a significant main effect of age, syllable length, and phonological complexity on repetition accuracy. Test– retest and interrater scoring showed high levels of reliability. Conclusion: The task described in this study offers an objective delivery of recorded stimuli that engages young children and provides high compliance rates. The task is inexpensive, requires minimal training, and can be adapted to other languages.
S
children, particularly under age 3 years, and tests that can be administered quickly are advantageous due to shorter attention spans. In this age group, direct assessments frequently have high noncompliance rates, so parental reports are often utilized as an alternative source of information about language abilities (Fenson et al., 1993; Sachse & Von Suchodoletz, 2008). The main disadvantage of indirect parental reports and checklists is that parents might underestimate or overestimate their children’s skills. Semistructured play interviews have been suggested as an alternative to formal assessment or parental reports (Brassard & Boehm, 2011; Lahey, 1988). Although assessment through play can be informative, analysis and interpretation can be timeconsuming and require the supervision of an experienced clinician. The aim of this study was to develop a NWR task for children between 2 and 6 years of age that can be easily administered, may be analyzed with minimal training, and can yield high levels of compliance. The relationship between language skills and NWR performance in young children has been shown in a number of studies. Hoff, Core, and Bridges (2008) reported significant correlations between vocabulary percentiles and NWR accuracy in two studies with children ages 1;8 to 2;0 (years; months), r(13) = .72, p < .001 and r(19) = .53, p < .01, respectively, and Stokes and Klee (2009) showed that NWR
uccessful assessments of children’s psycholinguistic processing and learning skills minimize the influence of prior experience and/or environmental factors on the resulting scores. Nonword repetition (NWR) tasks are seen as being less affected by environmental factors than traditional language tests (Campbell, Dollaghan, Needleman, & Janosky, 1997; Engel, Santos, & Gathercole, 2008). NWR tasks have also proven to be effective clinical markers for language impairment in a number of languages (English: see Coady & Evans, 2008 for review; Italian: Bortolini et al., 2006; Spanish: Girbau & Schwartz, 2007; Icelandic: Thordardottir, 2008; Slovak: Kapalková, Polišenská, & Vicenová, 2013) and have shown promise as a clinical tool (Archibald, 2008). Another important aspect of assessment is the opportunity to provide early identification of language deficits (e.g., Bercow, 2008; Rossetti, 2001). However, it is often challenging to assess the language abilities of younger
a
University of Manchester, United Kingdom Comenius University, Bratislava, Slovakia
b
Correspondence to Kamila Polišenská: [email protected] Editor: Rhea Paul Associate Editor: Stephanie Stokes Received January 16, 2013 Revision received May 17, 2013 Accepted August 19, 2013 DOI: 10.1044/1092-4388(2013/13-0014)
1060
Key Words: children, assessment, screening, phonology
Disclosure: The authors have declared that no competing interests existed at the time of publication.
Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014 • A American Speech-Language-Hearing Association
in English-speaking 2-year-olds distinguished late talkers from typically developing (TD) children, possibly due to the close links between phonological processing and lexical development. Similarly, NWR performance in Englishspeaking 2- to 3-year-olds was related to their morphosyntactic skills at ages 4 to 5 years, r(145) = .32, p < .001 (Chiat & Roy, 2008). These findings suggest that NWR performance in 2-year-old children is informative and can predict broader language skills, but potentially inconsistent delivery resulting from the use of nonrecorded stimuli and high noncompliance rates have remained a concern. Hoff et al.’s first NWR experiment with 16 TD children ages 1;9 to 2;0 had an impressive noncompliance rate of 6%, but the second attempt with the same age group (1;8–2;0) but different set of stimuli yielded a noncompliance rate of 22%, with six out of 27 children failing to complete the task. Stokes and Klee (2009) reported noncompliance rates of 20% and 27% in two versions of their Test of Early Nonword Repetition with a large sample of 2-year-old children, and an even higher noncompliance rate was reported in a sample of 2-year-old TD Dutch-speaking children tested by Zamuner (2009), with 37 out of 101 children (37%) not finishing the NWR task. Studies with samples including slightly older children reported better cooperation figures. Roy and Chiat (2004) carried out a study with 2- to 4-year-olds and reported a loss of 11% children due to noncooperation. Seeff-Gabriel, Chiat, and Roy (2008) tested children ages 2;0–5;11 and reported a refusal rate of 5.3% on their NWR and word repetition task administered live. The majority of children (90%) who refused to cooperate were children under 3. It has been suggested that noncompliance might be related to deficit (Hoff et al., 2008). Stokes and Klee (2009) showed that children who failed to complete their NWR task scored significantly lower on the MacArthur-Bates Communicative Development Inventory (CDI) and also on receptive and expressive vocabulary tasks than the children who completed the NWR task. However, it is unlikely that the proportion of children with a deficit would be as high as suggested by noncompliance rates of over 20% as reported in some of the studies discussed above, and it remains unclear which nonresponding children have a deficit and which nonrespondents are just unwilling to cooperate. We are not examining the reasons behind noncompliance but are attempting to increase the number of children who complete the task in order to reduce nonrespondent bias. In addition, studies on 2-year-old children have so far all opted to use live presentation rather than recorded stimuli in the belief that live presentation will lead to increased engagement and therefore higher completion rates. Live presentation is more flexible and may well assist in increasing engagement through greater opportunity for interaction, but it does mean that the delivery will be less consistent, and there is a greater likelihood of production errors by the tester. Our main aim was to develop a methodology for language assessments that could utilize the consistency of recorded nonword stimuli to minimize research bias while still effectively engaging children as young as 2 years. We also wanted to establish validity and reliability of this newly developed task.
An additional aim was to examine whether our NWR task was influenced by environmental factors such as socioeconomic status (SES). Campbell et al. (1997) tested school-aged boys from minority and majority backgrounds on a “knowledge dependent” measure that relied heavily on vocabulary and also “processing dependent” measures such as a Nonword Repetition Task (NRT; Campbell, Needleman, Riess, & Tobin, 1995). The study found that boys from minority backgrounds scored significantly lower than boys from majority backgrounds on the knowledge dependent measure but did not differ on the NRT, leading to the conclusion that processing dependent measures are not biased against minority speakers. More recently, a study by Engel et al. (2008) investigated the effect of Brazilian children’s SES on performance on vocabulary and NWR tasks. Children from the lower SES group scored significantly lower than children from the higher SES group on receptive and expressive vocabulary tests, but there was no significant difference on NWR scores. Reynolds and Fish (2010) reported similar findings for low-SES rural Appalachian children who scored lower compared to population scores on traditional language tests, but their mean nonword score was almost identical to the nonminority group from Campbell et al. (1997). These findings collectively support the idea that NWR tasks are less influenced by prior knowledge than traditional language tests. However, this conclusion has been called into question by a recent study by Roy and Chiat (2013) who found that English-speaking children from a low SES background scored significantly lower than children from a high SES background on both repetition tasks and more traditional assessments. As a result, the present study also investigated the effect of SES on NWR performance.
Method Participants The study was conducted according to the Slovakian ethical guidelines for research with human subjects. Parents of children in 15 nurseries in Slovakia were provided with an information sheet and consent form. Four hundred and two parents consented for their child to take part in the study. Consenting parents were also asked to fill in a questionnaire about parental education, the child’s date of birth, and any known developmental disorders, hearing problems, or concerns about their child’s development. Information about parental education was not always completed, and therefore this part of the data is sometimes missing (see the Results section for more detail). Only children who spoke Slovak as a first language and had no known developmental disorders were included in the analyses. This meant that 11 children were excluded: three children had a clinical diagnosis of a developmental disorder, six children did not speak Slovak as their first language, and information about two children’s date of birth was incomplete. Therefore, the final sample consisted of 391 children. From this sample, seven children failed to complete the task. Nurseries in both rural and urban areas were contacted. The final sample
Polišenská & Kapalková: Child Compliance on a Nonword Repetition Task
1061
consisted of 61 children (16%) from nurseries in a rural area and 323 children (84%) from nurseries in urban areas. A majority of the children (77%) attended state nurseries and 23% attended private nurseries. Table 1 shows the participants’ characteristics and the noncompliance rate for each age group and the total sample. A child was classified as noncompliant if she or he refused to attempt the task at all. Once the child started the test, nonresponses were very rare, and all children who started the test completed it.
presented on a laptop computer in a quiet room during a session that lasted about 5 min. The NWR task was embedded in a story:
Materials
The story was told in simple sentences with no sentence exceeding five words in the Slovak version. The simple structure and short sentences were chosen to ensure that even the youngest children could comprehend the story and instructions. There were two practice items and 26 nonword items in the full test. The nonwords were presented in a fixed random order. The sound was presented over external speakers connected to the laptop. All of the items were recorded in a professional sound studio in Bratislava and were edited using Audacity sound software. The sound files were embedded in a PowerPoint file and accompanied by the pictures presented in Figure 1. The experimenter controlled when each nonword was played. After a child repeated or attempted to repeat the item, a bead appeared on the necklace. Each item was played a maximum of two times. Every bead appeared with an animated effect and stayed on the screen, allowing the children to see their progress. In the end, all 26 beads appeared, and Hugo and Nina were shown smiling with the completed necklace for their Mom. All of the children’s responses were scored online and also recorded on a digital audio recorder. At the end of the session, the investigator verbally praised each child (rather than, for example, giving stickers); the child was shown a slide with the completed necklace they had made. The whole experiment is presented as a video in the supplemental materials. An editable PowerPoint presentation is available on request from the first author.
The task consisted of 26 nonwords with varying syllable length and phonological complexity (for a full set of items, see the Appendix). Some items were taken from a task developed by Kapalková et al. (2013) and new items were also created to control for phonological complexity. Items with CV, CVC, VC structures were considered phonologically simple, whereas CCV, CCVC, CVCC items were considered phonologically complex. There were four monosyllabic items: two phonologically simple without consonant clusters and two with a single consonant cluster either in the onset or coda. Similarly, the eight disyllabic items were divided between four phonologically simple and four complex stimuli. The same pattern also applied to the eight trisyllabic items: Four had a simple syllable structure and four a complex syllable structure. Four-syllable items were represented by one simple and one phonologically complex item, and this was also the case for five-syllable items. The total of 24 items could be categorized by length: 12 short items (1- and 2-syllable items) versus 12 longer items (3-, 4-, and 5-syllable items). The task also included two items with the consonants /l/ and /r/ as a syllable nucleus as the Slovak language allows syllabic /r/ and /l/ to form the nucleus of a syllable; such words are common in Slovak.
Procedure Each child was tested individually in a nursery by one of six qualified speech and language therapists who were all native speakers of Slovak. The tester showed the child the laptop and told him or her that it had games on it. The tester asked the child if he or she wanted to play a game on the computer, and if the child agreed, he or she was shown the NWR task (note that noncompliance occurred when children refused to play a game on the computer). The task was
Hugo and Nina made a necklace for their Mom. They used colorful beads. But the necklace broke. And the beads all fell on the floor. Hugo and Nina were very sad. But we can help them. With magic words, we can make a new necklace. When you repeat the magic word, a bead will appear on the necklace. Now let’s try it.
Scoring—Whole Items Each child’s responses were scored online during the experiment by the tester. All of the sessions were recorded, and a random subset of children from the recordings were also scored offline to determine reliability (see the Reliability section, below). Both the online and offline scoring used the same criteria: A correct repetition of a whole item earned
Table 1. Participants’ characteristics and noncompliance rate according to age groups. Participants
2-year-olds
3-year-olds
4-year-olds
5-year-olds
6-year-olds
Total
Recruited sample Noncompliance rate (%) Compliance rate (%) Final sample Mage in months (SD) Gender (F/M)
n = 28 14.3 85.7 n = 24 30.13 (3.51) 14/10
n = 63 4.8 95.2 n = 60 42.13 (2.76) 32/28
n = 113 0 100 n = 113 53.69 (3.79) 65/48
n = 115 0 100 n = 115 65.07 (3.52) 55/60
n = 72 0 100 n = 72 75.68 (3.51) 40/32
n = 391 1.79 98.21 n = 384 57.94 (13.4) 206/178
Note. F = female; M = male.
1062 Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014
Figure 1. Pictures accompanying the story in a nonword repetition task.
1 point and an incorrect repetition earned 0 points. A correct repetition meant that all consonants and all vowels were present and were repeated in the correct order. Any changes to the original sequences of phonemes were considered incorrect, and 0 points were awarded. The maximum number of points was 26. The aim of our task was to create an instrument that could be used in clinical practice for screening purposes—therefore featuring a quick and straightforward method of scoring—that could also be adopted by speech and language therapists, assistants, and teaching assistants (see the Reliability section). As it can be assumed that teaching assistants would have a limited knowledge of developmental speech errors, we opted for a strict scoring, which did not allow for these errors. Kapalková et al. (2013) explored several methods of scoring for Slovak and wholeitem scoring appeared to be clinically most relevant. Similar to other tests used in clinical practice (e.g., the Children’s Test of Nonword Repetition, Gathercole & Baddeley, 1996; The Preschool Repetition Test, Seeff-Gabriel et al., 2008), the whole-item scoring was preferred to other types of scoring, including phoneme percentage correct. This was also supported by a study by Roy and Chiat (2004), who compared the whole-item scoring and phoneme percentage correct on a sample of English-speaking children; they concluded that the scoring methods were equal at differentiating between TD and clinical samples, but whole-item scoring was quicker.
Results An analysis of variance (ANOVA) was conducted with type of nursery (two levels: state or private), nursery location (two levels: rural or urban), and gender (two levels: girls or boys) as between-subject factors. Age was entered into the analysis as a covariate: Previous studies (e.g., Chiat & Roy, 2007; Kapalková et al., 2013) showed that repetition
performance is age related. Type of nursery was not significantly associated with children’s performance on our NWR test, F(1, 377) = 1.70, p = .193, h = .004. Nursery location was also not significantly contributing to the differences, F(1, 377) = 0.06, p = .81, h = .000. Similarly, the difference between repetition performance of girls and boys was not significant, F(1, 377) = 0.81, p = .37, h =.002. Accordingly, all three factors were excluded from subsequent analyses. We also ran another analysis to assess whether maternal education had a significant effect. As previously mentioned, information on maternal education was not available for all of the children, so only 163 children were included in the analysis. The data on maternal education were divided into three categories: (a) children of mothers who did not pass the maturitná skúška (the Slovak national exam required for entry into university), (b) children of mothers who passed the maturitná skúška, and (c) children of mothers with a university degree. There was no main effect of maternal education, F(2, 159) = 1.64, p = .198, h = .02. The only significant factor in the between-subject analyses was age, and as a result, the data were collapsed across all nonsignificant effects for further analyses, with age remaining the only between-subject variable. A mixed ANOVA with Age as a between-subject variable and Length and Phonological Complexity as withinsubject variables was carried out to investigate whether the performance on the task replicated well-established findings reported in previous studies (e.g., Chiat & Roy, 2007; Jones, Tamburelli, Watson, Gobet, & Pine, 2010). The ANOVA revealed a significant main effect of Age, F(4, 379) = 65.03, p < .001, h = .41; length, F(1, 379) = 385.28, p < .001, h = .50; and phonological complexity, F(1, 379) = 79.04, p < .001, h = .17. There were also significant two-way interactions between Length × Age, F(4, 379) = 32.51, p .05 r(18) = .27, p > .05
Polišenská & Kapalková: Child Compliance on a Nonword Repetition Task
1065
the same or lower noncompliance rates as previous studies in children as young as 2 years. Archibald (2008, p. 25) suggested that “clinicians may unwittingly influence the nonword repetition performance of their clients by hyperarticulating more difficult or longer nonwords, or when clients are doing poorly.” The recorded stimuli in our study meant that participants were presented with identical prosody, speech rate, and volume. Consistent presentation of NWR stimuli is particularly important: Successful repetition is wholly dependent on accurate perception of the stimuli and the unfamiliar nature of the nonlexical items may be more likely to result in production errors by the tester. Our task contained 26 items, more than any other NWR task for young children, for example, two versions of the Test of Early Nonword Repetition (Stokes & Klee, 2009), which contain 12 or 16 nonword items; the Preschool Repetition Test by Seeff-Gabriel et al. (2008), which contains 18 nonword items; and Hoff et al. (2008) with nine nonwords in Study 1 and 12 nonwords in Study 2. In spite of the higher number of items in our study, all of the children who started the task completed the repetition of all 26 items. There are a number of advantages in having a task that can successfully incorporate more items. Having more items means that more individual measurements are possible, which in turn leads to greater testing reliability as a result of the reduced likelihood of measurement error. This is particularly important when the tester employs online whole-item scoring, which is less time-consuming and therefore offers better clinical utility. Our reliability measures suggested high level of agreement between online and offline scoring, and similar findings have been reported previously (Archibald, 2008; Seeff-Gabriel et al., 2008). Furthermore, having a task that can successfully incorporate more items allows for more items per variable, and this can again increase reliability. There is also the potential to use one set of items for different age groups but without older children scoring at ceiling. Our nonwords were designed to allow for the examination of common phonological processes. This was motivated by the previous finding that there is link between language impairment and systematic phonological errors on an NWR task (Kapalková et al., 2013). Having more items allows for a greater number of phonological processes to be examined and has the added advantage of allowing for the same process to be examined on multiple occasions. Because the task is recorded, a clinician can address the consistency of the phonological processes in greater detail rather than having to rely on online judgments. This is particularly useful in Slovak and other languages where no other expressive phonological assessments are available. Overall, the noncompliance rate was 1.79% for children ages 2 to 6 years. This reflects the simple and straightforward nature of the task, with its clear aim and easy to follow rules. The way that the children willingly participated without a direct reward such as stickers reinforces how the task appealed to the children. Our testers also reported that children seemed to like the fact that they could clearly see their progress: the beads on the screen showed how many nonwords had already been repeated and also how many
more items needed to be repeated in order to finish the necklace. Similar to Seeff-Gabriel et al. (2008), the noncompliance rate in our sample was age related, with the highest noncompliance rate in the youngest group. Our noncompliance rate of 14.3% was still lower than the rate reported in other studies with 2-year-olds (e.g., Stokes and Klee, 2009: 20%; Zamuner, 2009: 37%). A similar noncompliance rate (11%) was achieved by Chiat and Roy (2007) but with live presentation. Their study assessed both word and nonword repetition, and nonresponses were affected significantly by the lexical status of items. The rate of nonresponse for nonwords was actually significantly higher than the rate of nonresponse for words, suggesting that motivating children to produce nonwords is more challenging than word repetition. It should be noted that the noncompliance rate in our study reflects outright refusals, meaning that the children who refused to participate did not even begin the task. The children who failed to attempt the test also refused other activities suggested by the experimenter such as block building, playing with a toy car, or drawing. The nursery teachers also reported that these children showed less interest in language, rarely asked questions, or made comments. Although our task did engage even the youngest children in our sample, the noncompliance rate was still 14.3%. The youngest children do not cooperate for a number of different reasons such as short attention span, shyness, anxiety, and tiredness, and it is important to investigate the reasons behind children’s noncooperation. Recent work on this was reported in a study by Dohmen, Chiat, and Roy (2013), who explored the reasons for noncompliance in verbal and nonverbal imitation tasks in children ages 2 to 3. Further research could investigate the profiles of children who do not cooperate, for example by using parents’ or teachers’ questionnaires or spontaneous speech samples. The aim of our study was to decrease the refusal rate, thereby reducing the number of cases when clinicians need to interpret refusal. The task was proven to be reliable, with high levels of interrater reliability, and it was shown to produce a stable score over a short time gap after retesting children within a period of 2 weeks. In addition, the scores did not differ across children from different backgrounds or between girls or boys, confirming the finding of previous studies that NWR tasks are less influenced by environmental and demographic factors (Campbell et al., 1997). Maternal education was used in this study as the basis for the children’s SES, and it is possible that other factors (e.g., income or lack of qualifications) might lead to different results as found by Roy and Chiat (2013). Our task replicated previous findings related to the phonological properties of NWR tests in Germanic languages, such as syllable length and syllable complexity (e.g., Archibald & Gathercole, 2006; Jones et al., 2010). The reasons for the lack of the length effect, and the complexity effect in the group of 5-year-olds in particular, are unclear. This lack of difference was not related to any particular tester. The data samples for all three testers who administered the task to a group of 5-year-olds showed no significant difference. Further research is needed to clarify this finding.
1066 Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014
The NWR task used in this study is quick (approximately 5 min), easy to administer (minimal training required), and inexpensive (basic equipment needed and low cost of the test). This opens up the possibility that in addition to qualified clinicians, clinical assistants and teachers could make use of this test. Although the reliability on students’ scoring looks promising, further research is needed to assess teacheradministered reliability. In addition, further research is needed to assess online reliability of less experienced individuals such as assistants, as the current study only reported offline scoring. With this instrument, researchers have a delivery method that is easy to use, can cover multiple variables, and is suitable for 2- to 6-year-old children. In this study, we found correlations between NWR and other language assessments, and in particular, we demonstrated significant ties between grammatical skills and NWR performance and vocabulary and NWR scores, replicating earlier findings from English on significant correlations between CDI and NWR scores (e.g., Hoff et al., 2008). We were able to use a standardized assessment only in the youngest group; the issue of lack of standardized assessments has been discussed above. Upon the publication of our current results, including the video, we encourage researchers working in other languages to demonstrate concurrent validity. Another question for future research concerns the diagnostic accuracy of our NWR task. In the past, consistency was sacrificed in the hope of increasing the participation of younger participants, but our method provides both consistency of delivery and high compliance rates. Our task produces high levels of compliance while offering an objective and engaging delivery of recorded stimuli.
Acknowledgments This work was supported by contract no. APVV-0410-11 from the Slovak Research and Development Agency. The writing of this article was supported by COST Action IS0804, “Language Impairment in a Multilingual Society: Linguistic Patterns and the Road to Assessment” (http://www.bi-sli.org). We would also like to thank Zuzana Vicenová, Lenka Distlerová, Michaela Halamová, Martina Sussová, and Simona Mihaliková for their assistance with data collection, J. D. Fenton (www.edthis.com) for editing, and to all of the nursery staff members, parents, and children who participated.
References Archibald, L. M. D. (2008). The promise of nonword repetition as a clinical tool. Canadian Journal of Speech-Language Pathology and Audiology, 32, 21–28. Archibald, L. M. D., & Gathercole, S. E. (2006). Nonword repetition: A comparison of tests. Journal of Speech, Language, and Hearing Research, 49, 970–983. Bercow, J. (2008). The Bercow Report: A review of services for children and young people (0–19) with speech, language and communication needs. Nottingham, United Kingdom: DCSF Publications. Bortolini, U., Arfé, B., Caselli, C. M., Degasperi, L., Deevy, P., & Leonard, L. B. (2006). Clinical markers for specific language
impairment in Italian: the contribution of clitics and non-word repetition. International Journal of Language and Communication Disorders, 41, 695–712. Brassard, M. R., & Boehm, A. E. (2011). Preschool assessment: Principles and practices. New York, NY: Guilford Press. Campbell, T. F., Dollaghan, C., Needleman, H., & Janosky, J. (1997). Reducing bias in language assessment: Processingdependent measures. Journal of Speech, Language, and Hearing Research, 40, 519–525. Campbell, T. F., Needleman, H. L., Riess, J. A., & Tobin, M. J. (1995, June). Bone lead level and language processing performance. Poster session presented at the annual meeting of the Symposium on Research in Child Language Disorders, Madison, WI. Chiat, S., & Roy, P. (2007). The preschool repetition test: An evaluation of performance in typically developing and clinically referred children. Journal of Speech, Language, and Hearing Research, 50, 429–443. Chiat, S., & Roy, P. (2008). Early phonological and sociocognitive skills as predictors of later language and social communication outcomes. Journal of Child Psychology and Psychiatry, 49, 635–645. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, 284–290. Coady, J. A., & Evans, J. L. (2008). Uses and interpretations of nonword repetition tasks in children with and without specific language impairments (SLI). International Journal of Language and Communication Disorders, 43, 1–40. Conti-Ramsden, G., Botting, N., & Faragher, B. (2001). Psycholinguistic markers for specific language impairment (SLI). Journal of Child Psychology and Psychiatry and Allied Disciplines, 42, 741–748. Dohmen, A., Chiat, S., & Roy, P. (2013). Nonverbal imitation skills in children with specific language delay. Research in Developmental Disabilities, 34, 3288–3300. Engel, P. M. J., Santos, F. H., & Gathercole, S. E. (2008). Are working memory measures free of socioeconomic influence? Journal of Speech, Language, and Hearing Research, 51, 1580–1587. Fenson, L., Dale, P., Reznick, J. S., Thal, D., Bates, E., Hartung, J. P., Pethick, S., & Reilly, J. S. (1993). The MacArthur–Bates Communicative Development Inventories: User’s guide and technical manual. Baltimore, MD: Brookes. Gathercole, S. E., & Baddeley, A. D. (1996). The Children’s Test of Nonword Repetition. London, United Kingdom: The Psychological Corporation. Girbau, D., & Schwartz, R. G. (2007). Non-word repetition in Spanish-speaking children with specific language impairment (SLI). International Journal of Language and Communication Disorders, 42, 59–75. Hanulíková, A., & Hamann, S. (2010). Slovak. Journal of the International Phonetic Association, 40, 373–378. Hoff, E., Core, C., & Bridges, K. (2008). Non-word repetition assesses phonological memory and is related to vocabulary development in 20- to 24-month-olds. Journal of Child Language, 35, 903–916. Jones, G., Tamburelli, M., Watson, S. E., Gobet, F., & Pine, J. M. (2010). Lexicality and frequency in specific language impairment: Accuracy and error data from two nonword repetition tests. Journal of Speech, Language, and Hearing Research, 53, 1642–1655. Kapalková, S., Polišenská, K., & Vicenová, Z. (2013). Non-word repetition performance in Slovak-speaking children with and without SLI: Novel scoring methods. International Journal of Language and Communication Disorders, 48, 78–89.
Polišenská & Kapalková: Child Compliance on a Nonword Repetition Task
1067
Kapalková, S., Slančová, D., Bónová, I., Kesselová, J., & Mikulajová, M. (2010). Hodnotenie komunikačných schopností v ranom veku [Evaluation of communication skills in early age]. Bratislava, Slovakia: Slovenská asociácia logopédov. Lahey, M. (1988). Language disorders and language development. New York, NY: Macmillan. Reynolds, M. E., & Fish, M. (2010). Language skills in low-SES rural Appalachian children: Kindergarten to middle childhood. Journal of Applied Developmental Psychology, 31, 238–248. Rossetti, L. M. (2001). Communication intervention: Birth to three. San Diego, CA: Singular–Thompson Learning. Roy, P., & Chiat, S. (2004). A prosodically controlled word and nonword repetition task for 2- to 4-year-olds: Evidence from typically developing children. Journal of Speech, Language, and Hearing Research, 47, 223–234. Roy, P., & Chiat, S. (2013). Teasing apart disadvantage from disorder. In C. R. Marshall (Ed.), Current issues in developmental disorders (pp. 125–150). Hove, United Kingdom: Psychology Press. Sachse, S., & Von Suchodoletz, W. (2008). Early identification of language delay by direct language assessment or parent report? Journal of Developmental and Behavioral Pediatrics, 29, 34–41. Seeff-Gabriel, B., Chiat, S., & Roy, P. (2008). The early repetition battery. Oxford, United Kingdom: Harcourt Assessment.
Stokes, S. F., & Klee, T. (2009). The diagnostic accuracy of a new Test of Early Nonword Repetition for differentiating late talking and typically developing children. Journal of Speech, Language, and Hearing Research, 52, 872–882. Stokes, S. F., Wong, A. M.-Y., Fletcher, P., & Leonard, L. B. (2006). Nonword repetition and sentence repetition as clinical markers of specific language impairment: The case of Cantonese. Journal of Speech, Language, and Hearing Research, 49, 219–236. Streiner, D. L., & Norman, G. R. (1995). Health measurement scales: A practical guide to their development and use. Oxford, United Kingdom: Oxford University Press. Thordardottir, E. (2008). Language-specific effects of task demands on the manifestation of specific language impairment: A comparison of English and Icelandic. Journal of Speech, Language, and Hearing Research, 51, 922–937. van Daal, J., Verhoeven, L., van Leeuwe, J., & van Balkom, H. (2008). Working memory limitations in children with severe language impairment. Journal of Communication Disorders, 41, 85–107. Zamuner, T. S. (2009). Phonotactic probabilities at the onset of language development: Speech production and word position. Journal of Speech, Language, and Hearing Research, 52, 49–60.
Appendix Test Items Items
No cluster
Syllable structure
Cluster
Syllable structure
Practice trials
/mik/
CVC
/kra:ta/
CCV.CV
1-syllable
/sɛp/a /tʃu:t/
CVC CVC
/pisc/ /drup/a
CVCC CCVC
2-syllable
/dɔɲa/a /lumaj/a /ʒafi:t/b /i:stap/ /kr̩ɲɛ/
CV.CV CV.CVC CV.CVC VC.CVC CS.CV
/tʃ la:ta/ /plinɛt/ /zlixa/ /ɦa:kisc/
CCV.CV CCV.CVC CCV.CV CV.CVCC
3-syllable
/zieɟɪni/a /tutsifa:n/ /la:sminɔ/a /ɦɔpɛʃka/ /tʃɛvl ̩nɔ/
CV.CV.CV CV.CV.CVC CVC.CV.CV CV.CVC.CV CV.CS.CV
/spimaka/c /mu:grata/ /spɔxtaɲɛ/ /kvipɛta/
CCV.CV.CV CV.CCV.CV CCVC.CV.CV CCV.CV.CV
4-syllable
/bazɔtʃɛjɲi:k/d
CV.CV.CVC.CVC
/plɛbɔtɲina/
CCV.CVC.CV.CV
5-syllable
/ʃɔkamɔti:va/a
CV.CV.CV.CV.CV
/sputrabicɛʎkɔ/
CCV.CCV.CV.CVC.CV
a Items taken from a nonword repetition task described in Kapalková et al. (2013). bModified from /ʒafi:r/, from Kapalková et al. (2013). cModified from /spima:tʃ ka/, from Kapalková et al. (2013). dModified from /barɔtʃɛjɲi:k/, from Kapalková et al. (2013).
1068 Journal of Speech, Language, and Hearing Research • Vol. 57 • 1060–1068 • June 2014
Copyright of Journal of Speech, Language & Hearing Research is the property of American Speech-Language-Hearing Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
