Child Psychiatry Hum Dev DOI 10.1007/s10578-013-0430-6

ORIGINAL ARTICLE

Agreement Between Parents and Teachers on Preschool Children’s Behavior in a Clinical Sample with Externalizing Behavioral Problems Franziska Korsch • Franz Petermann

Ó Springer Science+Business Media New York 2013

Abstract An accurate interpretation of information obtained from multiple assessors is indispensible when complex diagnoses of behavioral problems in children need to be confirmed. The present study examined the similarity of parents and kindergarten teachers ratings on children’s behavior in a sample of 160 preschool children (a clinical group including 80 children with externalizing behavioral problems and a matched control group including 80 children). Behavioral problems were assessed using the SDQ, and the DISYPS-II questionnaires for ADHD and conduct disorders. The results revealed low levels of parent–teacher agreement for their ratings on the children’s behavior in both groups with the highest correlations in the non-clinical sample. Parent–teacher agreement did not differ significantly across the samples. Parent and teacher ratings correlated with the prevalence of externalizing disorders and were found to be almost independent of each other. The results highlight the importance of multiple informants and their independent influence within the diagnostic process. Keywords Psychological assessment  Agreement  Third-party reports  Preschool children  Externalizing behavioral problems

Introduction In recent years there has been an increasing interest in assessing externalizing behavioral disorders, especially in preschool children. Externalizing behavioral problems are F. Korsch (&)  F. Petermann Center of Clinical Psychology and Rehabilitation, University Bremen, Bremen, Germany e-mail: [email protected]

typically defined as the expression of extroverted problematic behaviors such as hyperactivity, impulsivity, aggression, or compulsivity, that interfere with a child’s general functioning for longer periods of time [1, 2]. Numerous studies were able to show the negative impact of externalizing behavioral problems on a child’s life, affecting peer relationships, social acceptance, and school achievements [2–15]. Given the high prevalence rates of externalizing behavioral problems in preschool years [16– 19] the question arises, how children at risk can be identified at an early age in a reliable manner [17, 20]. Preschool children are not yet able to provide realistic assessments about their own behavior [21–23]. Therefore, a key aspect in this growing research area is the accurate assessment of externalizing behavioral problems using multiple informants. For some externalizing behavior disorders such as ADHD, symptoms in different life contexts are required diagnostic criteria, while for others (e.g. conduct disorders), it could be important for the diagnostic process and for the planning of interventions to investigate problematic behavior in specific situations [4, 24]. Selection of Multiple Informants During the diagnostic process, clinicians cannot exclusively rely on their own observations but need to include other sources of information, like structured interviews or standardized questionnaires for behavioral assessment. There is a consensus in current research that although information obtained from one parent seems to be sufficiently representative for the child’s behavior in home situations [25–27], behavioral problems in other life situations could remain unidentified if only parental information is considered [28]. That is, behavioral data should be based on dissimilar sources to obtain a complete picture of

123

Child Psychiatry Hum Dev

a child’s behavioral patterns, leading to cross-contextual and multi-informant information becoming standard in clinical assessments [4, 24, 29–35]. In preschool samples, establishments like kindergartens represent a main source of behavioral information apart from the parents’ assessments. Given that kindergarten teachers spend a lot of time with the children, they often have good opportunities to observe a child’s social interactions in a larger group context [36]. Parents, however, typically do not have much insight in their children0 s behavior during kindergarten time.

behavior. A closer analysis of several socio-demographic variables (sex, mothers’ education, net income, migration status) failed to reveal any significant impact of those variables on the parent–teacher agreement except for the children’s age, with significantly lower intercorrelations for 3-year old compared to 6-year old children. Kuschel et al. [44] also examined the agreement between parents and teacher ratings for children with severe behavioral problems, but could not confirm greater similarity of parent and kindergarten teacher ratings for this group of children. Predictive Value

Informant Discrepancies Due to the complexity of behavioral assessments, it still needs to be clarified, whether different sources and informants can provide consistent information about a preschool child’s behavior. There still is no gold-standard in dealing with informant discrepancies and combining dissimilar assessments in a diagnostic process. A considerable amount of literature has been published on school-age children, showing only poor to moderate similarity between parents’ and teachers’ assessment with parents reporting problematic behavior more frequently and more severe [4, 28, 29, 37–39]. So far, however, it is much less clear if this is also the case in preschool children, especially in young children with clinically severe problematic behavior. Findings in community samples seem to be in accordance with studies examining older children of school age. Dinnebeil et al. [40] recently examined parent and teacher ratings in an American kindergarten sample and found just low levels of parent–teacher agreement. They found that teachers rate children’s problematic behavior significantly lower than parents do, and so were able to confirm the results of other community sample studies showing that parents report overall more problematic behavior than teachers do with only low similarity among the ratings [29, 41–43]. These findings are in accordance with the results reported by Winsler and Wallace [43], who analyzed the degree of agreement between parental and teacher ratings on 47 preschool children who were normally developing. In general, they found moderate levels of agreement (-.09 to .38) with poor inter-rater correlations for social skills (-.09 to .27) and moderate inter-rater correlations for externalizing behaviors (.29 to .38). They also found differences in the mean ratings, showing that parents reported significantly more problematic behavior than teachers did. Kuschel et al. [44] examined parent– teacher ratings for 310 German preschool children using the CBCL 1‘-5 and the C-TRF 1‘-5 and reported poor to moderate agreements ranging from ICC = .07–.30. Again, differences in the mean ratings of parents and teachers showed that parents report significantly more problematic

123

The main aim of behavioral assessment is the detection of problematic behavior and the prediction of clinically apparent behavior and poor outcome. In their study, Verhulst, Koot, and Van der Ende [45] found that teachers’ assessment of children’s behavior tended to predict poor outcomes after 6 years (including behavioral problems and the use of the mental health service) better than parents assessment, although they did not find significant differences between the two predictions. While the presence of inter-rater disagreements seems not to represent a valid predictor for adverse child outcome in clinical samples [46], studies on school-age children in community samples showed parents’ and teachers’ assessments to have both a high predictive value, with the use of combined ratings seeming to be preferable [47–49]. Clinical samples at school-age indicate that parents’ and teachers’ assessments both seem to predict psychiatric disorders [50] and poor outcome [51] with a fair degree of accuracy in the clinical setting. All these findings show that the combination of multiple data seems to be a better predictor of clinically apparent behavior than information from one informant alone regarding children of school-age, but there still is insufficient data on whether these results can be generalized for the assessment of younger children’s behavior. The Present Study Even though multiple assessments are frequently included in the modern diagnostic process, only limited data is available concerning the parent–teacher agreement for preschoolers with clinically apparent behavioral problems. The authors have only found one appropriate study including a preschool sample and explicitly examined the similarity between parent and kindergarten teacher ratings for higher symptom ranges, but not in a clinical sample. No study with case–control design was found to examine the similarity of agreement within a clinical preschool sample and a healthy control group. The present study was designed to expand on the present literature by testing the similarity in parent–teacher assessments in a case–control

Child Psychiatry Hum Dev

design with a clinical preschool sample. Furthermore, this paper attempts to show the significance of a combined examination of multiple-informants assessments in a diagnostic process with preschool children. In particular, we aimed at clarifying the following issues. According to the results previously reported [40, 41] we expected parents to report significantly greater behavioral problems in their children than kindergarten teachers would, especially in the clinical group, whereas no significant difference should appear for the ratings of prosocial behaviors. In due consideration of the main characteristics and cross-situational symptoms of externalizing behavioral problems [1, 2, 4, 24], we suggested parents and kindergarten teacher ratings to be more similar in the clinical sample than in healthy controls. Additionally, it was also intended to clarify, whether one could expect the prevalence of externalizing behavioral problems to be closer related to parents or to kindergarten teacher ratings and if either parents or teachers ratings could be redundant for predicting the presence of externalizing behavioral problems.

Method Participants and Recruitment The clinical sample was selected from a pool of children in a psychological follow-up examination of the earlier health examination for school entry in Bremen, Germany. This health examination for school entry is a general health and developmental check up performed by the local health authority. According to the respective state laws of Germany, all preschool children first have to pass this examination before entering primary school. Within this examination, the German state of Bremen implemented the Strengths and Difficulties Questionnaire [SDQ; 52] as a behavioral screening instrument in 2010. Children showing clinically apparent screening scores in the SDQ are given the opportunity to take part in a psychological follow-up examination [20]. The participation in this follow-up examination is voluntary and has no further impact on the school entry process. Clinical procedure was based on the ethical standards of the European Federation of Psychologists’ Association (EFPA) and the German Association of Psychology (Deutsche Gesellschaft fu¨r Psychologie; DGPs). We assured freedom of consent, self determination, no use of deception, and debriefing of the results. Parents were informed about the confidentiality of their personal data and all of them gave their consent for the scientific use of their data. According to the German Research Council (Deutsche Forschungsgemeinschaft; DFG) an ethic statement is only needed when the participants in the study are at risk or harm,

when the assessment is associated with physical and emotional distress or when the participants are not fully informed about the assessment procedures and aims of the study. Since this was not the case in our study, no approval of an ethical review committee was needed. For the clinical sample, we collected data from the psychological follow-up examinations in the years 2011 and 2012, including children with diagnosed or subclinical hyperkinetic disorders (ICD10 F90) or conduct and oppositional defiant disorders (ICD10 F91) according to the International Classification of Diseases—10th Version (ICD-10; [53]). The clinical sample consisted of 80 school starters (32.5 % girls and 67.5 % boys) aged between 68 and 86 months, and consisted of the diagnoses disturbance of activity and attention (ICD-10 F90.0; 49 %; N = 39), hyperkinetic conduct disorder (ICD10 F90.1; 30.0 %; N = 24), and oppositional defiant disorder (ICD-10 F91.3; 21 %; N = 17). With respect to the case–control design of the present study, we recruited a total of 80 children without any history of externalizing behavioral problems for the control sample. This was done by using newspaper advertisements in the city of Bremen and by laying out information material in kindergartens and pediatricians. All parents were given detailed information about the study procedure, voluntary participation conditions, and confidentiality of their data. Parents filled in a consent form to agree to the participation of their child in the present study and gave their consent for the scientific use of the data. Finally, the control sample consisted of 80 school starters without any history of externalizing behavioral problems. This sample was matched with the clinical sample for sex, age, and the number of years of the maternal school education. Those variables were selected for matching in order to avoid confounded results, since they have already been proven to influence parent and teacher ratings [32, 54– 57]. Furthermore, differences in family education could influence the understanding of the questionnaires and therefore alter the results. As shown in Table 1, the clinical and control samples did not significantly differ in their gender distributions (v2 (1) = .000; p = 1.00), nor did ages (v2 (21) = 22.094; p = .39), or the years of maternal school education (v2 (3) = 1.882; p = .597) significantly differ across both samples.

Table 1 Demographic statistics for the two samples matched by sex, age, and maternal education N

Sex

Age in months

Maternal education

Female

Male

M

SD

M

SD

Clinical

80

26

54

75.70

4.40

10.35

1.31

Control Total

80 160

26 52

54 108

75.54 76.62

4.70 4.53

10.10 10.23

1.42 1.37

123

Child Psychiatry Hum Dev

Measures Parents and kindergarten teachers filled in the Strengths and Difficulties Questionnaire [SDQ; 47, 52] as well as the symptom lists for hyperkinetic and conduct or oppositional defiant disorders of the diagnostic system for mental disorders in children and adolescents based on DSM-IV and ICD-10 criteria, German rating scale [Diagnostik-System fu¨r psychische Sto¨rungen nach ICD-10 und DSM-IV fu¨r Kinder und Jugendliche; DISYPS-II; 58]. Further instruments which were also used within the same clinical health evaluation for school starters were not considered in the current investigation. Data was collected during the last 4 months before starting school. Because all participating children had visited kindergarten for at least 2 years, it can be assumed that the kindergarten teachers did know the children well enough to complete the questionnaires adequately. Behavioral Screening The Strengths and Difficulties Questionnaire [SDQ; 36, 52] was implemented as a behavioral screening instrument in its German version. It is a 25-item screening questionnaire for self- and third-party (parents and teachers) reports on behaviors, emotions, and relationships of children between 5 and 16 years of age. The 25 items are equally associated with 5 scales: emotional problems, conduct problems, hyperactivity, peer problems, and prosocial behavior. Each item requires a rating on a 3-point Likert-type scale (not true, somewhat true, and certainly true). The item scores for each scale are added together and scaled to values between 0 and 10. The scores achieved on emotional problems, conduct problems, hyperactivity, and peer problems are added together to obtain scaled scores of the total difficulty ranging from 0 to 40. For this difficulty score as well as for all subscales except for the prosocial behavior, a higher score indicates more severe behavioral problems, and are classified as normal, borderline, or abnormal. We used the SDQ total sum score and the SDQ prosocial behavior score for further analyses. Externalizing Problematic Behavior Two questionnaires of the diagnostic system for psychical disorders according to ICD-10 and DSM-IV for children and adolescents [DISYPS-II; 58] were used to assess externalizing problematic behavior. The DISYPS-II is a German compilation of questionnaires for self- and thirdparty reports based on ICD-10 and DSM-IV diagnostic criteria, including the most common psychological disorders in children and adolescents. For the present study, the

123

DISYPS-II questionnaires for preschool ADHD (in German: ADHS-V) and conduct or oppositional defiant disorders (in German: SSV) were selected to assess the two forms of externalizing behavioral problems in young children. The ADHS-V consists of 19, the SSV of 25 items. Each item is rated on a 4-point Likert-type scale. Although each single questionnaire includes several subscales, total scores can be calculated by adding all item scores. For all scales a higher score indicates more severe behavioral problems. The ADHS-V and the SSV total sum score were both included in the following analyses. Statistical Analyses Data was administrated and statistically analyzed using IBM SPSS 20 for Windows. Measures of central tendency were used to describe parents and kindergarten teacher ratings on the SDQ total difficulty score, the SDQ prosocial behavior scale, and the total sum scores on the DISYPS-II ADHS-V and SSV questionnaires. Nonparametric tests were conducted, since Kolmogorov–Smirnov tests revealed deviations from the normal distribution for the data of all scales except for the SDQ total difficulties score (D (202) = .076; p = .026) and Levene’s test showed that the variances were significantly different in the two groups on the ADHS-V total score in ratings and in the SSV total scores of the kindergarten teachers. Because parents and kindergarten teachers assessed the same children (dependent testing), Wilcoxon signed-rank tests were chosen in order to compare parent–teacher ratings within groups. Furthermore, Spearman’s correlation coefficient (Spearman’s rho) was calculated for all relevant scores to evaluate different levels of similarity of parent and teacher ratings. Following common rules of thumb, Spearman’s rho below .30 were classified as poor, up to .50 as fair, and Spearman’s rho up to .80 as moderately strong, and greater or equal to .80 as strong relationships [59]. Fisher’s transformation was used for pair-wise comparisons of correlation coefficients. Furthermore, to clarify if either parents or kindergarten teacher ratings could be redundant for predicting the presence of externalizing behavioral problems, a binary logistic regression analysis with a forced entry method was conducted with the prevalence of a diagnosis as the categorical dependent variable (yes or no) and clinical apparent scores in parents and kindergarten teacher assessments as predictors (present or absent). According to the test manuals, cut-off scores at the 10th percentile for the SDQ total score [60] and the ADHS-V total sum score [58] were used to define clinical apparent scores. Data obtained from the SSV were not included in these analyses due to missing teacher norms for preschool age.

Child Psychiatry Hum Dev

Results Differences in Parents’ and Teachers’ Assessments First we examined, whether parents report more severe behavioral problem scores than kindergarten teachers. Figure 1 shows the boxplots of parents and teachers in the SDQ total difficulties score, the SDQ prosocial behavior scale, the ADHS-V sum scale, and the SSV sum scale for both sample groups. The parent and teacher ratings and similarities between these ratings in the clinical sample are illustrated in Table 2. Concerning children with externalizing behavioral problems, parents reported significantly more prosocial behavior in the SDQ (Mdif = 1.85; T = 429.00; p = .000; d = 0.91) with a large effect size, and the ADHS-V total sum score (Mdif = 3.93; T = 1135.00; p = .043; d = 0.32) with medium effect size. No significant differences were found in the SDQ total difficulties score (Mdif = 0.02; T = 1346.00; p = .980; d = 0.00), or the

SSV total sum score (Mdif = 0.28; T = 1531.50; p = .964; d = 0.03). In the control sample (see Table 3), parents showed significantly higher ratings on the SDQ prosocial behavior scale (Mdif = 0.93; T = 642.00; p = .003; d = 0.42), ADHS-V total sum score (Mdif = 3.61; T = 899.00; p = .002; d = 0.36), and SSV total sum score (Mdif = 3.39; T = 724.50; p = .001; d = 0.52) with medium effect sizes, but no significant differences were found between the SDQ total difficulties scores of parents and teachers (Mdif = 1.21; T = 1135.50; p = .089; d = 0.20). Similarity Between Parent and Teacher Ratings In the second part of the results, we take a closer look at the similarity between parent and kindergarten teacher assessments. Analysis showed overall poor similarity between parent and teacher assessments with the highest correlation coefficients for the ADHS-V total sum score (Spearman’s rho = .25) and for the SDQ total difficulties

Fig. 1 Boxplots of parent’s and kindergarten teacher’s assessments on the Strength and Difficulties Questionnaire (SDQ; total difficulties score and prosocial behavior scale) and the DISYPS-II questionnaires for preschool ADHD (ADHS-V; total score), and the DISYPS-II questionnaires for conduct disorders (SSV; total score) in the clinical (N = 80) and the control sample (N = 80)

Table 2 Parent and teacher assessments and agreements in the clinical sample (N = 80) Parents M SDQ

Total difficulties score Prosocial behavior

Teachers SD

M

Wilcoxon signed-rank test SD

Mdif

T

p

Spearman’s rho

d

16.46

5.43

16.44

6.62

0.02

1346.00

.980

0.00

.18

7.48

1.79

5.63

2.26

1.85

429.00

.000

0.91

.04

ADHS-V

Sum score

28.07

11.74

24.14

13.07

3.93

1135.00

.043

0.32

.25*

SSV

Sum score

14.68

9.53

14.40

9.36

0.28

1531.50

.964

0.03

.18

* p \ .05, d: Cohen’s d, SDQ: Strengths and Difficulties Questionnaire, ADHS-V: DISYPS-II questionnaires for preschool ADHD, SSV: DISYPS-II questionnaires for conduct disorders

123

Child Psychiatry Hum Dev Table 3 Parent and teacher assessments and agreements in the control sample (N = 80) Parents M SDQ

Total difficulties score Prosocial behavior

Teachers SD

M

Wilcoxon signed-rank test SD

Mdif

T

p

d

Spearman’s rho

10.16

5.59

8.95

6.28

1.21

1135.50

.089

0.20

7.98

1.99

7.05

2.43

0.93

642.00

.003

0.42

.29* .27*

ADHS-V

Sum score

13.46

9.82

9.85

10.42

3.61

899.00

.002

0.36

.29*

SSV

Sum score

7.43

7.41

4.04

5.57

3.39

724.50

.001

0.52

.17

* p \ .05; ** p \ .01; *** p \ .001, d: Cohen’s d, SDQ: Strengths and Difficulties Questionnaire, ADHS-V: DISYPS-II questionnaires for preschool ADHD, SSV: DISYPS-II questionnaires for conduct disorders

score (Spearman’s rho = .29) in the clinical group (see Table 2). As Table 3 indicates, correlation coefficients turned out to be the highest for ADHS-V total sum score (Spearman’s rho = .29) in the control sample. These results are consistent with our assumptions regarding poor agreement between parents and kindergarten teachers in both groups. In a next step, we wanted to examine if the clinical group showed significantly better agreement compared to the control group. Contrary to our expectations, the comparisons of the parent–teacher correlations across the clinical and control sample showed no significant differences in the agreement on the SDQ total difficulties score (Spearman’s rho (parents) = .18; Spearman’s rho (teachers) = .29; z-statistic = -0.72; p = .47), the SDQ prosocial behavior scale (Spearman’s rho (parents) = .04; Spearman’s rho (teachers) = .27; z = -1.47; p = .14), the ADHS-V total sum score (Spearman’s rho (parents) = .25; Spearman’s rho (teachers) = .29; z = -0.27; p = .79), and the SSV total sum score (Spearman’s rho (parents) = .18; Spearman’s rho (teachers) = .17; z = 0.06; p = .95). Presence and Prediction of Problematic Behavior When inspecting of correlations between the parent and teacher ratings with the later diagnosis, first we tested if the assessment of the parents or the kindergarten teachers were closer related to the diagnosis. Parents and teacher ratings in all questionnaires both significantly correlated with the presence of a diagnosed or presumed externalizing behavioral disorder on all scales. Correlations for parents’ ratings ranged from poor in the SDQ prosocial behavior (Spearman’s rho = -.16; p \ .05), over fair in the SSV total sum score (Spearman’s rho = .46; p \ .01), to moderate in the SDQ total difficulties score (Spearman’s rho = .51; p \ .01), and the ADHS-V total sum score (Spearman’s rho = .59; p \ .01). Whereas kindergarten teachers’ ratings showed fair correlation only in the SDQ prosocial behavior scale (Spearman’s rho = -.30; p \ .01) and moderate correlations on the SDQ total difficulties score (Spearman’s rho = .52; p \ .01), the ADHS-V sum score (Spearman’s rho = .53; p \ .01), and

123

the SSV total sum score (Spearman’s rho = .59; p \ .01). No significant differences between the parents’ and kindergarten teachers’ correlations with the diagnosis were found on the SDQ total difficulties score (z = -0.12; p = .95), the SDQ prosocial behavior scale (z = 1.31; p = .19), the ADHS-V total sum score (z = 0.78; p = .44), and the SSV total sum score (z = -1.60; p = .11). So it can be summarized that parent and teacher assessments both seem to be significantly related to the later diagnosis given. Finally, we wanted to show that neither parent nor kindergarten teacher assessments are redundant in the diagnostic process. Table 4 presents the results of the binary logistic regression analysis. Parents and teacher ratings for child behavior in the SDQ (R2 = .272 (Cox & Snell), .363 (Nagelkerke); v2 (2) = 50.843, p = .000) and ADHS-V (R2 = .304 (Cox & Snell), .405 (Nagelkerke); v2 (2) = 57.995, p = .000) contributed significantly to the prediction of the presence of externalizing behavioral problems. The proportion of correct classification was 76.9 % in the model including both SDQ assessments, and 75.6 % in the model using both ADHS-V ratings. According to Nagelkerke’s R2, the model including parents and teacher assessments explained 36.3 % of the variance of the presence of externalizing behavioral problems, the model including parent and teacher ADHS-V assessments explained 40.5 %. No predictor was excluded during the analysis, leading to the expected assumption, that the best model in the prediction of a later diagnosis is the one including both ratings.

Discussion The current study is focusing on the similarity in parent and kindergarten teacher assessments of young children with externalizing behavioral problems. Of particular interest was the question, if there are significant differences in the assessments of this clinical group compared to the agreement between parents and teachers in a matched and healthy control sample.

Child Psychiatry Hum Dev Table 4 Binary logistic regression analysis of parents and teacher assessments predicting the presence of externalizing behavioral problems Predictor variables

B (SE)

95 % CL for odds ratio

p

Lower

Odds ratio

Upper

2.50 2.69

5.52 5.78

12.21 12.44

SDQ Step 1 Parents assessment Teachers assessment

1.709 (.405) 1.754 (.391)

.000 .000

ADHS-V Step 1 Parents assessment

1.967 (.513)

2.62

7.15

19.53

.000

Teachers assessment

2.053 (.443)

3.27

7.79

18.55

.000

SDQ: Strengths and Difficulties Questionnaire, ADHS-V: DISYPS-II questionnaires for preschool ADHD

First we examined, whether parents reported higher behavioral problem scores than kindergarten teachers. In accordance with our expectations, parents and teachers showed overall higher ratings in the assessment of problematic behavior in both groups, even though, the differences between the parents’ and teachers’ ratings were only significant in the ADHD symptoms in both groups, and the conduct behavior in the control group. These results are concordant with recent findings and confirm that parents seem to report more problematic externalizing behavior [17, 40, 44]. One explanation for the parents tending to report more problematic behavior than kindergarten teachers do, could be, that teachers are less likely to report problematic behavior in school-age children unless the child shows co-morbid peer or behavioral problems [42, 61]. However, contrary to expectations our results showed parents to report significantly more prosocial behavior than kindergarten teachers did in both groups. This may be because parents met the clinicians personally and filled in the questionnaires in a clinical context, while teachers had no personal contact to the clinicians. So the parents’ report of prosocial behavior could be due to a social desirability bias. According to the correlation between parent’s and teacher’s assessments, as expected, we found only poor agreement between parents and kindergarten teachers in both groups. In the clinical group, agreements for parent and teacher ratings on problematic and prosocial behavior, ADHD symptoms, and compulsive behavior turned out to be quite low, with the best agreements for the ADHD symptom ratings. In the control sample, higher but again poor correlations were found for the problematic behavior screening and the assessment of prosocial behavior and ADHD symptoms. This findings are consistent with those reported in the latest meta-analysis of Achenbach et al. [62]. They analyzed parent–teacher agreements in 119 studies and found an average correlation coefficient of .27.

However, low similarity between parent and teacher ratings for prosocial behavior ratings found in our data (.04 in the clinical sample; .27 in the control sample) was not compatible with the findings of a meta-analysis conducted by Renk et al. [63], who found a mean r of .42 in the parent– teacher agreements on social competence in early childhood. Generally, the poor agreement in our study agrees with other, more recent, results for the social and behavioral ratings of children in preschool care [40, 41] and for the assessment of ADHD symptoms [24, 29]. Nevertheless, it has to be noted that other studies found slightly higher correlation coefficients (.29 to .38) in the assessment of externalizing behavioral problems in a US preschool sample of normally developing children [55]. Contrary to our expectations, a comparison between the agreement among parents and kindergarten teachers in both groups showed high similarity. Parent–teacher agreements in the clinical group did not significantly differ from those found in the non-clinical group on all scales. So even in a clinical group with externalizing behavioral problems, who would be expected to show significant problematic behavior in all areas of a child’s life, parents and teachers did not agree better in their behavioral assessment than they did in a healthy control sample. Concerning the question whether parents’ or teachers’ assessment was closer related to the prevalence of externalizing behavioral problems, single and independent correlation analyses confirmed that parent and kindergarten teacher assessments were both significantly related with the diagnostic status, even though correlations turned out to be poor to moderate. According to the low correlation, it should be taken into account that correlations depend on the variances. So, if there were larger variance in one of the groups, the correlations could be assumed to be higher. However, no significant differences were found between the correlations of the parents’ and teachers’ assessment with the presence of externalizing behavioral problems.

123

Child Psychiatry Hum Dev

Provided that clinically relevant scores had been achieved on the SDQ total difficulties measure, the odds of being diagnosed with an externalizing behavioral disorder were 5.5 times higher with a corresponding parents rating and 5.8 times higher if the teacher’s rating showed a clinically apparent result. Clinically apparent scores on the ADHD symptom questionnaire increased the odds of a diagnosis to 7.1 times in the parents’ assessments and to 7.8 times in the kindergarten teachers’ assessments. Although our findings do not support previous research that suggested parents’ ratings to make better predictions on future child diagnoses than teachers’ ratings [46], the current results are consistent with those studies which found parents and teacher reports both to be important and not interchangeable assessors [32]. As Brown et al. [28] could show in their study, parents failed to detect half of those children that were considered by teachers to be likely to show clinical apparent behavior. Other research confirms, that parents and teachers seem to identify different children in primary care as seriously disturbed [64]. These results strongly support the assumption, that both assessments might have an independent impact on the diagnostic process of externalizing behavioral problems and informant discrepancies should not only be interpreted as measurement errors but can themselves contain important information for the diagnostic process [22, 65, 66]. Explanations for Informant Discrepancies Over time numerous attempts have been made to explain the dissimilarity of parents and teacher ratings concerning the assessment of behavioral problems in young children [34]. First, it is not uncommon that children’s behavior differs depending on the situational context and the person they are interacting with. One reason for this may be that children learn that depending on the given context, the same behaviour could be interpreted as problematic or appropriate. Some argue that the poor agreement among parents and teachers can be attributed to this situation specific behavior [24, 29]. Apart from that actually different behavior, the assessors also have different possibilities to observe specific behavior in their given context. So while parents interact with their children in a family setting they might have to handle other problematic situations with their children at home (e.g. monitoring family rules) than kindergarten teachers do in an educational situation (e.g., reaching educational goals). Preschool children tend to spend less time with their teachers than school-age children. On the one hand, this decreases the teachers’ opportunity to observe children’s behavior but on the other hand, lowers the parent’s possibilities to observe peer interaction [41, 42, 67]. Additionally, parents often only have a small number of other children of a similar age in

123

the same setting to compare their own child with [67], whereas kindergarten teachers see large groups of children and are exposed to a larger spectrum of group behavior and could have a better eye for age-inappropriate developments [42]. Additionally, apart from the different situations, the observers are able to observe children in, the observers themselves can have different observing angles based on the roles the assessors fulfill in the different contexts and the associated responsibilities [24, 29, 41, 42]. Furthermore, current research indicates that parents and teachers have the tendency to base their assessments on behavior in an unfamiliar context on the situation they normally observe the child in (parents-home and teachers-school) [42, 68]. Taken together, differences between informants from different contexts may provide unique information to our understanding of a child’s behavior [4, 22, 33, 65, 66]. Furthermore, a parent’s reporting bias according to clinical or control group membership should be taken into consideration. This is, parents in a clinical sample could be suggested to possibly exaggerate their child’s problematic behaviors when they want their child to receive therapeutic treatment, whereas parents in the control group might be more likely to underestimate their child’s inappropriate behaviors to avoid stigmatization [51, 54]. Due to the suggested inter-rater disagreements concerning the assessment of externalizing behavioral problems, the question arises, how to deal best with informant discrepancies. To date, however, there is no gold standard available for the integration of multiple assessments in the diagnostic process [4]. Inter-rater disagreements do not represent valid predictors for adverse child outcome [46] and during the past few years, research and practice learned that there is more information contained in assessment discrepancies than just measurement errors. For example, disagreement among raters may tell us something about the cause of problematic behavior. On the one hand, higherrated problematic behavior in teachers’ reports might indicate that parents do not sufficiently identify certain behavior as problematic or that they use different standards compared to teachers. Higher values in parents’ ratings on the other hand could point to home-specific problems caused by deficits in parenting styles [46]. Moreover, low agreements could disclose the presence of conflicts in the teacher-child relationship, especially when boys with externalizing behavioral problems are involved [54].

Limitations and Future Directions Despite the strengths of our study, including a clinical preschool sample with a matched control group and the use of the same questionnaires for parents’ and kindergarten teachers’ assessment, caution should be taken in

Child Psychiatry Hum Dev

generalizing these findings with respect to normal populations. Since the questionnaires applied were part of the complex diagnostic process on which the diagnoses are based, several measures could be suggested as being confounded. Longitudinal data, however, would enable a better evaluation of the extent to which parents and teacher ratings could influence the diagnostic process. In our clinical sample we were able to demonstrate the importance of different observers for the diagnostic process of clinically apparent externalizing behavior in a preschool sample. In future research it would be of considerable interest to analyze the structure of multiple assessments in other disorders and syndromes. For a better understanding of the complex structure of inter-rater disagreement, further work needs to be done to determine if dissimilarity among parents and teachers in their assessments is caused by the different interaction styles of the adults when dealing with the children which triggers truly different context-specific behavior in the children [41, 69]. Finally, the examination of the kindergarten teacher’s individual characteristics could be of interest. For example, Berg-Nielsen et al. [54] were able to show that a problematic child-teacher interaction can lower the agreement between parents and kindergarten teachers within behavioral assessments. Concerning further rater characteristics, it would be quite interesting to examine whether gender differences, such as mothers reporting slightly higher problem scores compared to fathers in interparental assessments, could also be found in teacher–teacher assessments.

Summary This study, which was the first to examine the similarity of agreement between parents and kindergarten teachers on behavior assessments in a preschool case–control design, compared the agreement between parents and teachers in their behavioral assessment in a clinical group with known externalizing behavior problems in comparison to the assessments in a healthy control sample. Contrary to expectation, we found a high similarity in the lack of agreement within the clinical and the control sample, with both groups showing poor agreement in their assessments. These results make meaningful contributions to the topic of the similarity of parent and teacher ratings by showing that there is very little agreement between these two informants in a clinical preschool sample with externalizing behavioral problems. Finally we were able to present evidence for the importance of both informants’ ratings in the diagnostic process. Despite the lack of similarity in their assessment, the consideration of both informants seems to be the best model in predicting the presence of externalizing behavioral problems and the combination of parent and

kindergarten teacher reports should be considered as being more sensitive than one single report. Conflict of interest of interest.

The authors declare that they have no conflict

References 1. Garnefski N, Kraaija V, van Etten M (2005) Specificity of relations between adolescents’ cognitive emotion regulation strategies and internalizing and externalizing psychopathology. J Adolesc 28:619–631 2. Henricsson L, Rydell A-M (2006) Children with behaviour problems: the influence of social competence and social relations on problem stability, school achievement and peer acceptance across the first six years of school. Infant Child Dev 15:347–366 3. Abikoff HB, Jensen PS, Arnold LLE, Hoza B, Hechtman L, Pollack S et al (2002) Observed classroom behavior of children with ADHD: relationship to gender and comorbidity. J Abnorm Child Psychol 30:349–359 4. Dirks MA, Boyle MH, Georgiades K (2011) Psychological symptoms in youth and later socioeconomic functioning: do associations vary by informant? J Clin Child Adolesc Psychol 40:10–22 5. Kerr DC, Lunkenheimer ES, Olson SL (2007) Assessment of child problem behaviors by multiple informants: a longitudinal study from preschool to school entry. J Child Psychol Psyc 48:967–975 6. Graziano PA, Geffken GR, McNamara JP (2011) Attypical behavior and comorbid externalizing symptoms equally predict children with attention-deficit/hyperactivity disorder’s social functioning. Child Psychiatry Hum Dev 42:377–389 7. Sciberras E, Ohan J, Anderson V (2012) Bullying and peer victimisation in adolescent girls with attention-deficit/hyperactivity disorder. Child Psychiatry Hum Dev 43:254–270 8. Tseng W-L, Kawabata Y, Gau SS-F (2011) Social adjustment among taiwanese children with symptoms of ADHD, ODD, and ADHD comorbid with ODD. Child Psychiatry Hum Dev 42: 134–151 9. Harty SC, Miller CJ, Newcorn JH, Halperin JM (2009) Adolescents with childhood ADHD and comorbid disruptove behavior disorders: aggression, anger, and hostility. Child Psychiatry Hum Dev 40:85–97 10. Korsch F, Petermann U, Schmidt S, Petermann F (2013) Kognitive, sprachliche, motorische und sozial-emotionale Defizite bei verhaltensauffa¨lligen Schulanfa¨ngern. [Cognitive, linguistic, motoric, and social deficits in schoolstarters with behavioral disorders]. Prax Kinderpsychol Kinderpsychiatr 62:405–419 11. Willcutt E, Betjemann R, Wadsworth S, Samuelsson S, Corley R, DeFries J et al (2007) Preschool twin study of the relation between attention-deficit/hyperactivity disorder and prereading skills. Read Writ 20:103–125 12. Wadepohl H, Koglin U, Vonderlin E, Petermann F (2011) Fo¨rderung sozial-emotionaler Kompetenzen im Kindergarten. Evaluation eines pra¨ventiven Verhaltenstrainings [Promoting social-emotional competence in preschool: evaluation of a preventive behavioral training program]. Kindh Entwickl 20:219–228 13. Ekorna˚s B, Heimann M, Tjus T, Heyerdahl S, Lundervold AJ (2011) Primary school children’s peer relationships: discrepancies in self-perceived social acceptance in children with emotional or behavioral disorders. J Soc Clin Psychol 30:570–582 14. Schreyer I, Petermann F, Petermann U (2011) Verhaltensund Entwicklungsauffa¨lligkeiten im Vorschulalter [Behavior

123

Child Psychiatry Hum Dev

15.

16.

17.

18.

19.

20.

21. 22.

23.

24.

25.

26.

27.

28.

29.

30.

problems and developmental delay in preschool children: to what extent is the quality of life of mothers and children affected?]. Z Gesundheitspsychol 19:147–156 Schuchardt K, Grube D, Ma¨hler C (2013) ‘‘Schwierige Kinder’’ von Anfang an? Aufmerksamkeitsprobleme als Risikofaktor fu¨r die Schulfa¨higkeit [‘‘Difficult children’’ right from the start? Attention problems as a risk for school readiness]. Kindh Entwickl 22:217–223 Baker BL, Neece CL, Fenning RM, Crnic KA, Blacher J (2010) Mental disorders in five-year-old children with or without developmental delay: focus on ADHD. J Clin Child Adolesc Psychol 39:492–505 Elberling H, Linneberg A, Olsen E, Goodman R, Skovgaard A (2010) The prevalence of SDQ-measured mental health problems at age 5–7 years and identification of predictors from birth to preschool age in a Danish birth cohort: The Copenhagen Child Cohort 2000. Eur Child Adolesc Psychiatry 19:725–735 Fuchs S, Klein AM, Otto Y, von Klitzing K (2013) Prevalence of emotional and behavioral symptoms and their impact on daily life activities in a community sample of 3 to 5-year-old children. Child Psychiatry Hum Dev 44:493–503 Gawrilow C, Petermann F, Schuchardt K (2013) ADHS im Vorschulalter: Subgruppen, Diagnostik und gezielte Therapieansa¨tze [ADHD at a preschool age: Subgroups, diagnostic, and specific treatments approaches]. Kindh Entwickl 22:189–192 Korsch F, Petermann F (2012) Fru¨herkennung von Verhaltenssto¨rungen durch die schula¨rztliche Eingangsuntersuchung [Identification of early childhood behavioral disorders by German preschool medical examinations]. Prax Kinderpsychol Kinderpsychiatr 61:691–705 Angold A, Egger HL (2007) Preschool psychopathology: lessons for the lifespan. J Child Psychol Psyc 48:961–966 Achenbach TM (2011) Commentary: definitely more than measurement error: but how should we understand and deal with informant discrepancies? J Clin Child Adolesc Psychol 40:80–86 van Mare´es N, Petermann F (2009) Der Bullying- und Viktimisierungsfragebogen fu¨r Kinder (BVF-K): Konstruktion und Analyse eines Verfahrens zur Erhebung von Bullying im Vorund Grundschulalter [The bullying and victimization questionnaire for children (BVF-K): Construction and analysis of an instrument for the assessment of bullying in kindergarten and primary school]. Prax Kinderpsychol Kinderpsychiatr 58:96–109 Re AM, Cornoldi C (2009) Two new rating scales for assessment of ADHD symptoms in Italian preschool children. J Atten Disord 12:532–539 Schroeder JF, Hood MM, Hughes HM (2010) Inter-parent agreement on the syndrome scales of the child behavior checklist (CBCL): correspondence and discrepancies. J Child Fam Stud 19:646–653 Langberg JM, Epstein JN, Simon JO, Loren REA (2012) Parent agreement on ratings of children‘s attention deficit/hyperactivity disorder and broadband externalizing behaviors. J Emot Behav Disord 18:41–50 Dave´ S, Nazareth I, Senior R, Sherr L (2008) A comparison of father and mother report of child behaviour on the Strength and Difficulties Questionnaire. Child Psychiatry Hum Dev 39:399–413 Brown JD, Wissow LS, Gadomski A, Zachary C, Bartlett E, Horn I (2006) Parent and teacher mental health ratings of children using primary care services: inter-rater agreement and implications for mental health screening. Ambul Pediatr 6:347–351 Graves SL, Blake J, Kim ES (2012) Differences in parent and teacher ratings of preschool problem behavior in a national sample: the significance of gender and SES. J Early Interv 34:151–165 Salbach-Andrae H, Klinkowski N, Lenz K, Lehmkuhl U (2009) Agreement between youth-reported and parent-reported

123

31.

32.

33.

34.

35.

36.

37.

38.

39. 40.

41.

42.

43.

44.

45.

46.

47.

psychopathology in a referred sample. Eur Child Adolesc Psychiatry 18:136–143 Kim-Cohen J, Arseneault L, Caspi A, Tomas MP, Taylor A, Moffitt TE (2005) Validity of DSM-IV conduct disorder in 4‘–5year-old children: a longitudinal epidemiological study. Am J Psychiatry 162:1108–1117 Phillips BM, Lonigan CJ (2010) Child and informant influences on behavioral ratings of preschool children. Psychol Schools 47:374–390 Tasse´ M, Lecavalier L (2000) Comparing parent and teacher ratings of social competence and problem behaviors. Am J Ment Retard 105:252–259 De Los Reyes A, Kazdin AE (2005) Informant discrepancies in the assessment of childhood psychopathology: a critical review, theoretical framework, and recommendations for further study. Psychol Bull 131:483–509 De Los Reyes A (2013) Strategic objectives for improving understanding of informant discrepancies in developmental psychopathology research. Dev Psychopathol 25:669–682 Koglin U, Barquero B, Mayer H, Scheithauer H, Petermann F (2007) Deutsche Version des Strengths and Difficulties Questionnaire (SDQ-Deu) [German version of the Strengths and Difficulties Questionnaire (T4-16—SDQ): psychometric quality of the teacher version for preschoolers]. Diagnostica 53:175–183 Park HI, Kim JM, Park YC, Kim SH, Ahn DH (2010) Agreement between parents and teachers on school children’s emotional/ behavioral problems. J Korean Acad Child Adolesc Psychiatry 21:161–167 Pourahmadi E, Jalali M, Alizadeh Sahraei H (2012) Agreement between parent and teacher for behavioral and emotional problems in a normative sample of Tehran children aged 7–11. Eur Psychiatry 27(Supplement 1):1 Gomez R (2007) Australian parent and teacher ratings of the DSM-IV ADHD symptoms. J Atten Disord 2007:17–27 Dinnebeil LA, Sawyer BE, Logan J, Dynia JM, Cancio E, Justice LM (2013) Influences on the congruence between parents’ and teachers’ ratings of young children’s social skills and problem behaviors. Early Child Res Q 28:144–152 Cai X, Kaiser AP, Hancock TB (2004) Parent and teacher agreement on child behavior checklist items in a sample of preschoolers from low-income and predominantly african american families. J Clin Child Adolesc Psychol 33:303–312 Strickland J, Hopkins J, Keenan K (2012) Mother-teacher agreement on preschoolers’ symptoms of ODD and CD: does context matter? J Abnorm Child Psychol 40:933–943 Winsler A, Wallace GL (2002) Behavior problems and social skills in preschool children: parent–teacher agreement and relations with classroom observations. Early Educ Dev 13:41–58 Kuschel A, Heinrichs N, Bertram H, Naumann S, Hahlweg K (2007) Wie gut stimmen Eltern und Erzieherinnen in der Beurteilung von Verhaltensproblemen bei Kindergartenkindern u¨berein? [How well do parents and teachers’ reports agree on behaviour problems in pre-schoolaged children?]. Z Kinder Jugendpsychiatr Psychother 35:51–58 Verhulst F, Koot H, Ende J (1994) Differential predictive value of parents’ and teachers’ reports of children’s problem behaviors: a longitudinal study. J Abnorm Child Psychol 22:531–546 Ferdinand RF, Jvd Ende, Verhulst FC (2007) Parent–teacher disagreement regarding behavioral and emotional problems in referred children is not a risk factor for poor outcome. Eur Child Adolesc Psychiatry 16:121–127 Goodman R, Ford T, Simmons H, Gatward R, Meltzer H (2003) Using the strength and difficulties questionnaire (SDQ) to screen for child psychiatric disorders in a community sample. Int Rev Psychiatr 1515:166–172

Child Psychiatry Hum Dev 48. Owens J, Hoza B (2003) Diagnostic utility of DSM-IV-TR symptoms in the prediction of DSM-IV-TR ADHD subtypes and ODD. J Atten Disord 7:11–27 49. van Dulmen MHM, Egeland B (2011) Analyzing multiple informant data on child and adolescent behavior problems: predictive validity and comparison of aggregation procedures. Int J Behav Dev 35:84–92 50. Goodman R, Renfrew D, Mullick M (2000) Predicting type of psychiatric disorder from strengths and difficulties questionnaire (SDQ) scores in child mental health clinics in London and Dhaka. Eur Child Adolesc Psychiatry 9:129–134 51. Ferdinand RF, Hoogerheide KN, Van Der Ende J, Visser JH, Koot HM, Kasius MC et al (2003) The role of the clinician: threeyear predictive value of parents’, teachers’, and clinicians’ judgment of childhood psychopathology. J Child Psychol Psyc 44:867–876 52. Goodman R (1997) The strengths and difficulties Questionnaire: a research note. J Child Psychol Psyc 38:581–586 53. WHO (1993) The ICD-10 classification of mental and behavioural disorders: diagnostic criteria for research. World Health Organization, Geneva 54. Berg-Nielsen TS, Solheim E, Belsky J, Wichstrom L (2012) Preschoolers’ psychosocial problems: in the eyes of the beholder? Adding teacher characteristics as determinants of discrepant parent–teacher reports. Child Psychiatry Hum Dev 43:393–413 55. Gagnon C, Vitaro F, Tremblay RE (1992) Parent–teacher agreement on kindergarteners’ behavior problems: a research note. J Child Psychol Psyc 33:1255–1261 56. Harland R, Reijneveld SA, Brugman E, Verloove-Vanhorick SP, Verhulst FC (2002) Family factors and life events as risk factors for behavioural and emotional problems in children. Eur Child Adolesc Psychiatry 11:176–184 57. Javo C, Ronning JA, Heyerdahl S, Rudmin FW (2004) Parenting correlates of child behavior problems in a lultiethic community sample of preschool children in northern Norway. Eur Child Adolesc Psychiatry 13:8–18 58. Do¨pfner M, Go¨rtz-Dorten A, Lehmkuhl G, Breuer D, Goletz H (2008) Diagnostik-System fu¨r psychische Sto¨rungen nach ICD-10 und DSM-IV fu¨r Kinder und Jugendliche [Diagnostic system for psychical disorders for children and adolescents according to ICD-10 and DSM-IV]. Huber, Bern

59. Chan YH (2003) Biostatistics 104: correlation analysis. Singapore Med J 44:614–619 60. Woerner W, Becker A, Friedrich C, Rothenberger A, Klasen H, Goodman R (2002) Normierung und Evaluation der deutschen Elternversion des Strengths and Difficulties Questionnaire (SDQ): Ergebnisse einer repra¨sentativen Felderhebung [Normative data and evaluation of the German parent-rated Strength and Difficulties Questionnaire (SDQ): results of a representative field study]. Z Kinder Jugendpsychiatr Psychother 30:105–112 61. Drugli MB, Larsson B, Clifford G, Fossum S (2007) Pervasive and non-pervasive conduct problems in a clinic sample aged 4–8 years: child, family and day-care/school factors. Scand J Educ Res 51:547–559 62. Achenbach TM, McConaughy SH, Howell CT (1987) Child/ adolescent behavioral and emotional problems: implications of cross-informant correlations for situational specificy. Psychol Bull 101:213–232 63. Renk K, Phares V (2004) Cross-informant ratings of social competence in children and adolescents. Clin Psychol Rev 24:239–254 64. Munkvold L, Lundervold A, Lie SA, Manger T (2009) Should there be separate parent and teacher-based categories of ODD? Evidence from a general population. J Child Psychol Psyc 50:1264–1272 65. De Los Reyes A, Thomas SA, Goodman KL, Kundey SMA (2013) Principles underlying the use of multiple informants’ reports. Annu Rev Psychol 9:123–149 66. Hartley AG, Zakriski AL, Wright JC (2011) Probing the depths of informant discrepancies: contextual Influences on divergence and convergence. J Clin Child Adolesc Psychol 40:54–66 67. Rescorla LA, Achenbach TM, Ivanova MY, Bilenberg N, Bjarnadottir G, Denner S et al (2012) Behavioral/emotional problems of preschoolers: caregiver/teacher reports from 15 societies. J Emot Behav Disord 20:68–81 68. Nijs PA, Ferdinand R, Bruin E, Dekker MJ, Duijn C, Verhulst D (2004) Attention-deficit/hyperactivity disorder (ADHD): parents’ judgment about school, teachers’ judgment about home. Eur Child Adolesc Psychiatry 13:315–320 69. Petermann F, Petermann U (2013) Sto¨rung des Sozialverhaltens [Conduct disorder]. Kindh Entwickl 22:123–126

123