J Autism Dev Disord DOI 10.1007/s10803-014-2306-4

ORIGINAL PAPER

Comparing Diagnostic Outcomes of Autism Spectrum Disorder Using DSM-IV-TR and DSM-5 Criteria Elizabeth B. Harstad • Jason Fogler • Georgios Sideridis • Sarah Weas • Carrie Mauras William J. Barbaresi

•

Ó Springer Science+Business Media New York 2014

Abstract Controversy exists regarding the DSM-5 criteria for ASD. This study tested the psychometric properties of the DSM-5 model and determined how well it performed across different gender, IQ, and DSM-IV-TR sub-type, using clinically collected data on 227 subjects (median age = 3.95 years, majority had IQ [ 70). DSM-5 was psychometrically superior to the DSM-IV-TR model (Comparative Fit Index of 0.970 vs 0.879, respectively). Measurement invariance revealed good model fit across gender and IQ. Younger children tended to meet fewer diagnostic criteria. Those with autistic disorder were more likely to meet social communication and repetitive behaviors criteria (p \ .001) than those with PDD-NOS. DSM-5 is a robust model but will identify a different, albeit overlapping population of individuals compared to DSM-IV-TR. Keywords Autism spectrum disorder (ASD)  DSM-5  Confirmatory factor analysis  Measurement invariance

Introduction Autism is diagnosed clinically using diagnostic criteria described in the Diagnostic and Statistical Manual of Mental Disorders (DSM). The updated DSM, Fifth Edition (DSM-5; American Psychiatric Association 2013) was published in E. B. Harstad (&)  J. Fogler  G. Sideridis  S. Weas  C. Mauras  W. J. Barbaresi Division of Developmental Medicine, Boston Children’s Hospital, Boston, MA, USA e-mail: [email protected] E. B. Harstad  J. Fogler  G. Sideridis  C. Mauras  W. J. Barbaresi Harvard Medical School, Boston, MA, USA

May 2013, replacing the DSM, Fourth Edition-Text Revision (DSM-IV-TR; American Psychiatric Association 2000) with controversial changes made in the definition and criteria for autism (Lord and Jones 2012; McPartland et al. 2012). The DSM-IV-TR categorized autism symptoms into three domains (social interaction; communication; and restricted, repetitive behaviors) with diagnostic sub-types denoted: autistic disorder, pervasive developmental disorder-not otherwise specified (PDD-NOS) and Asperger’s disorder. Rett’s Disorder and Childhood Disintegrative Disorder are also included in DSM-IV-TR (American Psychiatric Association 2000). Rett’s Disorder is often associated with an underlying genetic etiology (Amir et al. 1999) and Childhood Disintegrative Disorder has physical symptoms, such as bowel and bladder dysfunction, that accompany significant regression in developmental skills (Charan 2012). In contrast, the DSM-5 consolidates the sub-types of autism into a single ‘‘autism spectrum disorder’’ (ASD) diagnosis and categorizes autism symptoms into only two domains: social communication and restricted, repetitive patterns of behavior, interests or activities (American Psychiatric Association 2013). A number of concerns have been voiced about the changes to the DSM criteria. These concerns include whether a new model for autism is necessary and if the new DSM-5 model for ASD will disproportionately exclude individuals with particular cognitive abilities, of certain ages, and with some specific levels of functioning (Kulage et al. 2014; McPartland et al. 2012). The shift from a three to two factor model for ASD arose because of studies indicating that the two factor structure is a better way to conceptualize ASD (Guthrie et al. 2013; Lord and Jones 2012; Mandy et al. 2012). Many studies have found that the DSM-IV-TR diagnostic criteria are vague and difficult to use reliably (Lecavalier et al. 2009; Mahoney et al. 1998). Within the DSM-IV-TR ASD model, the decision regarding

123

J Autism Dev Disord

whether a behavioral characteristic fit within the social or communication domain in DSM-IV-TR was often made arbitrarily. Therefore, combining these criteria into one social communication domain may potentially create a more robust model (Gotham et al. 2007). Throughout the development of the two factor DSM-5 ASD model, research has been conducted to evaluate how well the model describes autism symptoms (Norris et al. 2012). Mandy et al. (2012) and Guthrie et al. (2013) have both used confirmatory factor analyses to show that the DSM-5 model is superior to the DSM-IV-TR model. Their data were obtained from parent interview and structured clinical observations, respectively. Further research, employing both parent-provided history and behavioral observations that are routinely used to make clinical diagnoses, will be important to determine if DSM-5 continues to perform as a superior model in the clinical setting. Recent studies suggest that changes in ASD diagnostic criteria may increase the specificity of the diagnosis, but disproportionately exclude patients with higher intellectual functioning (McPartland et al. 2012; Taheri and Perry 2012), toddlers (Barton et al. 2013; Matson et al. 2012b), and those with DSM-IV-TR diagnoses of PDD-NOS or Asperger’s disorder (Gibbs et al. 2012; McPartland et al. 2012). McPartland et al. (2012) re-analyzed ASD symptoms among a sample of 933 participants in the DSM-IV field trial; 40 % (mostly those higher intellectual functioning, PDD-NOS, or Asperger’s disorder) did not meet the new DSM-5 ASD criteria. Matson et al. (2012b) evaluated children receiving Early Intervention services in Louisiana. The use of the DSM-5 criteria resulted in 47 % fewer toddlers being diagnosed with ASD, compared to the DSM-IV-TR criteria. The existing literature is primarily based on retrospective chart review and/or application of the ASD criteria in a research setting (Gibbs et al. 2012; Guthrie et al. 2013; McPartland et al. 2012). However, since ASD diagnoses are made in real time in clinical settings, it is important to better understand how the changing criteria may affect clinical practice. The objectives of the current study were to determine: (1) whether the three factor DSM-IV-TR ASD model or the two factor DSM-5 ASD model provides a better fit for clinically collected ASD data (as opposed to data collected in the context of a controlled research study); and (2) how well the DSM-5 model performs across heterogeneous populations based on gender, IQ, age, and diagnostic sub-type (autistic disorder, PDD-NOS, and Asperger’s disorder).

Methods Study Setting and Participants This study occurred in a multi-disciplinary developmentalbehavioral pediatric clinic that evaluates and treats children

123

with a range of developmental and behavioral conditions at a large academic medical center. Subjects in this study include all children and adolescents (ages 16 months to 18 years) seen for a new multi-disciplinary assessment in the developmental-behavioral pediatric clinic between October 2012–July 2013 for whom clinicians completed both DSM-IV-TR and DSM-5 ASD symptom checklists indicating that ASD was a diagnostic consideration. This developmental-behavioral pediatric clinic provides consultation and ongoing follow-up care for children and adolescents with developmental or behavioral conditions. The study was originally conceived as a Quality Improvement initiative to increase adherence to current diagnostic criteria and ease clinician transition to the new DSM-5 ASD criteria. Subsequent to the onset of this quality improvement initiative, we obtained Institutional Review Board approval for a de-identified analysis of DSM-IV-TR versus DSM-5 constructs of ASD. In October 2012, clinicians in the developmentalbehavioral pediatric clinic began using standardized checklists to record DSM-IV-TR (see Table 1a) and proposed DSM-5 (see Table 1b) ASD criteria for children seen for a multi-disciplinary diagnostic consultation. Clinicians used the 2011 DSM-5 ASD draft criteria; wording for each ASD symptom in this draft was identical to the item wording in the final published DSM-5. Clinicians were asked to record whether or not ASD was a diagnostic consideration. If an ASD diagnosis was considered, clinicians marked which DSM-IV-TR and DSM-5 criteria were met, if any. In the current study, we retrospectively reviewed these clinician-completed checklists and the accompanying medical record data for each subject. Diagnoses of autistic disorder, PDD-NOS, and Asperger’s disorder were categorized according to the established DSM-IV-TR criteria (American Psychiatric Association 2000). Specifically, to meet criteria for an autistic disorder diagnosis, subjects needed a total of at least 6 DSM-IV-TR criteria met, including at least 2 social interaction criteria, 1 communication criteria, and 1 restricted, repetitive behavior, interest or activity criteria. To meet criteria for PDDNOS, subjects needed to meet some DSM-IV-TR criteria, but did not meet the DSM-IV-TR specified threshold for autistic disorder or Asperger’s Disorder. To meet criteria for an Asperger’s disorder diagnosis, subjects needed to fulfill at least 2 social interaction criteria and 1 restricted, repetitive behavior, interest or activity criteria without having significant cognitive, developmental, or language delays. As part of regular clinical care, all subjects received a multi-disciplinary team assessment from a developmentalbehavioral pediatrician and child psychologist. Each team consisted of both a staff level board certified/board eligible developmental behavioral pediatrician and a staff doctoral

J Autism Dev Disord Table 1 (a) DSM-IV-TR and (b) DSM-5 checklist completed for team consultation visits Check if positive

DSM-IV-TR autism spectrum disorder diagnostic criteria

(a) 1. Qualitative impairment in social interaction, as manifested by at least 2 of the following 1. Marked impairment in the use of multiple nonverbal behaviors such as eye–eye gaze, facial expression, body postures, and gestures to regulate social interaction 2. Failure to develop peer relationships appropriate to developmental level 3. A lack of spontaneous seeking to share enjoyment, interests, or achievements with other people (e.g., by a lack of showing, bringing, or pointing out objects of interest) 4. Lack of social or emotional reciprocity 2. Qualitative impairments in communication as manifested by at least one of the following 1. Delay in, or total lack of, the development of spoken language (not accompanied by an attempt to compensate through alternative modes of communication such as gesture or mime) 2. In individuals with adequate speech, marked impairment in the ability to initiate or sustain a conversation with others 3. Stereotyped and repetitive use of language or idiosyncratic language 4. Lack of varied, spontaneous make-believe play or social imitative play appropriate to developmental level 3. Restricted repetitive and stereotyped patterns of behavior, interests, and activities, as manifested by at least one of the following 1. Encompassing preoccupation with one or more stereotyped and restricted patterns of interest that is abnormal either in intensity or focus 2. Apparently inflexible adherence to specific, nonfunctional routines or rituals 3. Stereotyped and repetitive motor mannerisms (e.g., hand or finger flapping or twisting, or complex whole body movements) 4. Persistent preoccupation with parts of objects Check if positive

DSM-5 autism spectrum disorder diagnostic criteria

(b) 1. Persistent deficits in social communication and social interaction across contexts, not accounted for by general developmental delays, and manifest by all 3 of the following 1. Deficits in social-emotional reciprocity; ranging from abnormal social approach and failure of normal back and forth conversation through reduced sharing of interests, emotions, and affect and response to total lack of initiation of social interaction 2. Deficits in nonverbal communicative behaviors used for social interaction; ranging from poorly integrated- verbal and nonverbal communication, through abnormalities in eye contact and body-language, or deficits in understanding and use of nonverbal communication, to total lack of facial expression or gestures 3. Deficits in developing and maintaining relationships, appropriate to developmental level (beyond those with caregivers); ranging from difficulties adjusting behavior to suit different social contexts through difficulties in sharing imaginative play and in making friends to an apparent absence of interest in people 2. Restricted, repetitive patterns of behavior, interests, or activities as manifested by at least two of the following 1. Stereotyped or repetitive speech, motor movements, or use of objects; (such as simple motor stereotypies, echolalia, repetitive use of objects, or idiosyncratic phrases) 2. Excessive adherence to routines, ritualized patterns of verbal or nonverbal behavior, or excessive resistance to change; (such as motoric rituals, insistence on same route or food, repetitive questioning or extreme distress at small changes) 3. Highly restricted, fixated interests that are abnormal in intensity or focus; (such as strong attachment to or preoccupation with unusual objects, excessively circumscribed or perseverative interests) 4. Hyper-or hypo-reactivity to sensory input or unusual interest in sensory aspects of environment; (such as apparent indifference to pain/heat/cold, adverse response to specific sounds or textures, excessive smelling or touching of objects, fascination with lights or spinning objects) DSM-IV-TR Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition-Text Revision, DSM-5 Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition

level licensed psychologist. Some teams included developmental-behavioral pediatric fellows and/or fellow level psychology trainees supervised closely in real time through

one-way observation windows by the staff level providers, who then led a team meeting to review findings from the assessment, formulate diagnoses and complete the DSM

123

J Autism Dev Disord

checklists as a team. The multi-disciplinary team assessment included collection of a thorough medical and developmental history and, in almost all cases, administration of a standardized measure to determine cognitive or developmental level. The cognitive/developmental testing included several different standardized tests (most commonly the Bayley Scales of Infant and Toddler Development, Third Edition (Bayley 2006) and the Differential Ability Scales, Second Edition (Elliott 2007). During the team assessment, the Autism Diagnostic Observation Schedule, Second Edition (ADOS-2; Lord et al. 2000) was administered if clinically indicated. At the end of each consultation visit, team members jointly discussed their assessment results, formulated diagnoses and treatment recommendations, and completed the DSM-IV-TR and proposed DSM-5 criteria checklists for all subjects for whom ASD was a diagnostic consideration. The following information was abstracted from each subject’s medical record: age, results of cognitive/developmental tests and ADOS-2 (if administered), diagnoses made during the team consultation, and item responses endorsed on the cliniciancompleted checklists for both DSM-IV-TR and DSM-5 ASD criteria.

Data Analysis Descriptive Statistics and Assessing for Clinician Rater Differences Frequencies, mean and median were used to describe the total sample, and sample separated by DSM-IV-TR and DSM-5 ASD diagnostic status. Chi square analyses were used to determine if differences existed between subjects diagnosed with ASD based on DSM-IV-TR versus DSM-5 criteria. A multilevel model was built to test the potential influence of inter-rater variability, given that many different pediatricians and psychologists completed the DSM-IVTR and DSM-5 checklists. This prerequisite analysis involved fitting a multilevel model (see ‘‘Appendix 1’’) to the data in order to evaluate whether the variability of DSM-5 scores is due to the differential team composition (pediatrician and psychologist), after accounting for the effects of participant age, gender, and DSM-IV-TR variance (so that the evaluation about team composition would be free of confounds). Results using robust standard errors indicated that the effects of team composition were null (b = 0.0129, p = .769). Thus, the different diagnostic teams were associated with a very small amount of variability. In other words, no specific diagnostic team was associated with being more or less likely to diagnose ASD based on the DSM-5 criteria, when compared to all of the other teams.

123

Confirmatory Factor Analysis Comparing DSM-IV-TR and DSM-5 ASD Models Confirmatory factor analysis (CFA) was employed to test the construct validity of DSM-IV-TR and DSM-5 models of ASD (Bollen 1989; Joreskog 1973). An initial power analysis established that ample levels of power were present to properly estimate the factor models. CFA evaluates the goodness of fit between an a priori statistical model, such as DSM-IV-TR versus DSM-5 ASD, and a null model in which there are no special distinctions between indicators (e.g., items measuring repetitive behaviors would be made to load on the same factor as items measuring social reciprocity). Several descriptive indices were employed to evaluate the discrepancy between the hypothetical and observed variance–covariance matrices due to the inability of the Chi square test statistic to properly evaluate model fit. Specifically, we used the Comparative Fit Index (CFI) known to be sensitive to relatively small sample sizes (Bentler 1990) and the presence of unreliability (Stuive et al. 2008) and the Tucker-Lewis Index (TLI; Tucker and Lewis 1973), often termed the Non-Normed Fit Index (Bentler and Bonett 1980), which is a relative fit index and reflects misfit in mean square errors (Widaman and Thompson 2003). For all indices values greater than .950 are considered adequate (Hu and Bentler 1995, 1998, 1999). Also, the root mean squared error of approximation (RMSEA) was employed among residual-based fit indices (Steiger 1990, 2000, 2007), for which valuesB 0.08 signify adequate fit. Because both of the above two indices have not been free from criticism (Kenny 2012; Marsh et al. 2005; Rigdon 1996) a residual-based fit index, i.e., the RMSEA was also implemented. The RMSEA has recently been the preferred index in evaluating CFA models because it is relatively unaffected by sample size (Raykov and Marcoulides 2000), it has a recommended range of acceptable values, it does not require a reference model, it makes reasonable adjustments for model length,1 and it provides easy to use confidence intervals (Loehlin 2004; Widaman and Thompson 2003). The Akaike Information Criteria (AIC; Akaike 1974, 1980) and Bayesian Information Criteria (BIC; Schwarz 1978) indices were used to compare DSM-IV-TR and DSM-5 models. These were used as employed as the only appropriate descriptive indices that can compare non-nested models as a function of model complexity based on the magnitude of the likelihood function. The primary goal of the Akaike Information Criteria (AIC) and Bayesian Information Criteria (BIC) is to avoid overfitting by penalizing more complex models, which will always be associated with improved fit compared to simple models. 1

In essence favoring more complex models (Breivik and Olsson 2001).

J Autism Dev Disord

The AIC index provides a penalty of two units for each additional parameter estimation, whereas the BIC penalises as a positive function of increasing sample size. The smaller the value of each index, the better the model fit. A corrected version of the BIC index (sample size adjusted BIC-SABIC) was used in the present study. This modification has been proposed as the uncorrected BIC may place too high a penalty on model complexity (see: Enders and Tofighi 2008; Tofghi and Enders 2007).2 Prior to performing the analyses below, a series of ancillary analyses evaluated the adequacy of the present sample to inferentially support the tested hypotheses. Initially, a power calculation was completed through simulation. Given the negative bias introduced by brief instruments in the estimation of power using the root mean squared error of approximation (RMSEA; MacCallum et al. 1996) a simulation study was run with estimates of factor loadings, residuals and factor correlations derived from the present sample’s estimates. Results indicated that a sample size of 200 participants was associated with stable parameters and accurate model fit. Specifically the mean bias in the estimation of the RMSEA was .003, and of the Chi square omnibus test .04. The bias in the estimation of factor loadings ranged between .008 and .01 (see ‘‘Appendix 2’’). Thus, ample levels of power were available across all models suggesting stability of model parameters and proper rejection of misfitted models using the present sample. These findings agree with recent simulation studies that have supported the adequacy of small sample sizes when implementing structural equation models (Sideridis et al. 2014). Measurement Invariance Testing of DSM-5 ASD Model Across Groups Measurement invariance refers to the overall stability of a hypothesized factor structure among different populations. It evaluates whether the DSM-5 two factor conceptualization of ASD can be consistently applied to any patient who might potentially meet ASD criteria. The invariance of the DSM-5 two factor structure was tested across various populations (gender, IQ, age, and diagnostic sub-type under DSM-IV2

The sample size adjusted BIC-SABIC was preferred to the corrected AIC (Hurvich and Tsai 1989) for the following reason: the AICc has proven useful for the time series autoregressive models for which it was originally developed. There is to date little evidence on its utility in other types of analyses (Brockwell and Davis 1991; McQuarrie and Tsai 1998). With small, less complex models and medium sample sizes, as was the present case, both AIC and AICc will generate similar estimates. Based on the early work of Rafterty (1995), Gignac and Watkins (2013) have recommended that effect sizes need to be suggested for AIC and BIC. They recommended that difference AIC/BIC values of 2, 6, 10 or [10 units reflect ‘‘weak’’, ‘‘positive’’, ‘‘strong’’, and ‘‘very strong’’ effects in favor of the simpler model.

TR). Three types of measurement invariance were tested (a) configural, (b) metric, and (c) scalar. Configural invariance, as depicted in Fig. 1, is satisfied when the same simple structure is validated across populations: Is the DSM5 model defined by the same factor structure in different patient populations, such as male and female? More restrictively, metric invariance, as depicted in Fig. 2, upper panel, involves the equivalence of both simple structures and the item’s factor loadings (Muthen and Muthen 2007): Is the definition of social communication and restrictive, repetitive domains that indicate ASD the same across populations? Lastly, scalar invariance, as depicted in Fig. 2, lower panel, involves the above constraints with the addition of the equivalence of item intercepts (Raykov 2005) which in the presence of dichotomously scored instruments are termed thresholds: Are each of the individual ASD criteria (three in the social communication domain and four in the repetitive behavior domain) met consistently across populations?

Results Participant Characteristics There were 227 subjects seen for multi-disciplinary assessment with DSM-IV-TR and DSM-5 checklists completed for whom ASD was a diagnostic consideration; 83.7 % were male; median age was 3.95 years (IQR = 4.57). The majority (54.2 %) of subjects were given a co-morbid psychiatric diagnosis; co-morbid diagnoses included Developmental Coordination Disorder (16.7 %), other medical conditions (14.5 %), Attention Deficit Hyperactivity Disorder (12.8 %), and Learning Disorder (6.6 %). Of the total sample, the DSMIV-TR breakdown of autism diagnoses was as follows: autistic disorder 50.4 %, PDD-NOS 16.1 %, Asperger’s disorder 2.2 %. There were no subjects diagnosed with Rett’s syndrome or Childhood Disintegrative Disorder. There were significant differences between subjects meeting DSM-IV-TR versus DSM-5 ASD criteria [v2(4) = 225.00, p \ .05]. Of the 156 subjects who met DSM-IV-TR criteria for an autism spectrum disorder, 23 % (N = 36) did not meet DSM-5 ASD criteria. There were no subjects who met DSM-5 ASD criteria but did not meet DSM-IV-TR ASD criteria. Children with DSM-IV-TR autistic disorder were significantly more likely to fulfill DSM-5 ASD criteria compared to children with DSMIV-TR PDD-NOS or Asperger’s disorder, as shown in Table 2. Among children with DSM-IV-TR ASD, there was a nonsignificant trend for children with IQ \ 70 and to be more likely to fulfill DSM-5 ASD criteria, compared to children with IQ [ 70. The effects of inter-rater variability on ASD diagnosis were non-significant for DSM-IV-TR [bRATER = 0.0589, t(216) = 0.761, p = 0.447] and DSM-5 criteria [bRATER = 0.0129, t(216) = 0.295, p = 0.769].

123

J Autism Dev Disord

Fig. 1 Hypothetical model for the measurement of configural invariance in DSM-5 conceptualization across two populations (e.g., boys versus girls separated by horizontal arrow). This tests if the DSM-5 model fits both genders, with no constraints placed on the model

Confirmatory Factor Analysis Comparing DSM-IV-TR and DSM-5 ASD Models Confirmatory factor analysis (CFA) was used to determine which model for ASD (DSM-IV-TR versus DSM-5) has better construct validity. The data were fit to each model separately and the models were then tested statistically using Akaike Information Criteria (AIC) and Bayesian Information Criteria (BIC). Table 3 shows that the data fit the conceptualization of the 2-factor DSM-5 model well (e.g., TLI = .958) and less so the three-factor DSM-IV-TR model (i.e., Fit indices below .900). The model superiority of the DSM-5 solution was evident across all fit indices, whereas the DSM-IV-TR conceptualization was not associated with a psychometrically acceptable model fit.

Measurement Invariance by Gender The DSM-5 ASD model fit the data well for both boys and girls at the levels of configural, metric, and scalar invariance. First, all measurement paths were significant for both groups. Secondly, the overall Chi square test was nonsignificant, suggesting that the factor structure fit both populations well. Metric invariance, positing the equivalence of factor loadings, was also satisfied across gender with superb model fit and a non-significant Chi square value. Lastly, scalar invariance was also satisfied, because the model in which item intercepts (thresholds) were constrained to be equal across groups was associated with good model fit. Measurement Invariance by Intelligence Quotient (IQ)

Measurement Invariance Testing DSM-5 ASD Model Across Groups To determine the relative ‘‘universality’’ of the DSM-5 model of ASD across subgroups of interest, measurement invariance was tested for the following independent variables: gender, IQ, age, and diagnostic sub-type under DSM-IV-TR (autistic disorder, PDD-NOS, and Asperger’s disorder). For the sections that follow, results are summarized in Table 4.

123

For children administered the Bayley Scales of Infant and Toddler Development, Third Edition the cognitive domain score was used as a proxy for the child’s IQ score, and IQ scores were categorized as B70 and [70, consistent with prior studies using this cut-off (Kent et al. 2013; McPartland et al. 2012). Configural, metric, and scalar invariance were all satisfied, suggesting that the DSM-5 model describes ASD well for those with an IQ B70 and [70. In

J Autism Dev Disord

Metric

Scalar

Fig. 2 Hypothetical model for the measurement of metric (top) and scalar (bottom) invariance in DSM-5 conceptualization across two populations (e.g., boys versus girls, separated by horizontal arrows). In metric invariance constraints are placed on the way that the individual criteria load onto the domains of social communication or

repetitive behaviors (termed equivalence of slopes). In scalar invariance, constraints are placed on both the way the individuals criteria load onto the domains as well as the intercepts associated with those factor loadings (termed equivalence of thresholds)

123

J Autism Dev Disord Table 2 Characteristics of the total sample and characteristics divided by DSM-IV-TR and DSM-5 autism spectrum disorder status Characteristics

Total sample (N = 227)

ASD by both DSM-IV-TR and DSM-5 (N = 156)

ASD by DSMIV-TR but not by DSM-5 (N = 36)

Test statistic (p value)

No ASD by either DSM-IVTR or DSM-5 (N = 71)

ASD with DSM-5 diagnosis but no DSM-IV-TRa (N = 0)

% Male

83.70

86.54

88.89

(0.78)*

77.14

–

5.25 (3.45)

4.75 (3.31)

4.7 (3.9)

-0.10

6.25 (3.4)

Age

t test:

Mean (SD)

–

Median (IQR)

3.95 (4.57)

3.59 (3.85)

3.57 (2.85)

(0.92)

5.57 (5.08)

% Cognitive level [70

76.55

72.26

86.11

v2: 3.47 (0.06)

85.71

–

% With ADOS administered

91.15

96.15

100

(0.34)*

78.57

–

% With following co-morbid conditions ADHD

12.78

6.41

8.33

v2: 0.29 (0.59)

27.14

–

Learning disorder

6.61

3.85

2.78

v2: 0.14 (0.70)

12.86

–

Developmental coordination disorder

16.74

12.18

16.67

v2: 0.88 (0.35)

27.14

–

Other medical

14.54

11.54

11.11

v2: 0.005 (0.92)

65.71

–

2

54.19

40.38

33.33

v : 0.96 (0.33)

25.71

–

Autistic disorder PDD-NOS

50.45 16.07

73.38 23.38

31.43 62.86

v2: 40.50 (\0.0001)

– –

– –

Asperger’s disorder

2.23

3.25

5.71

–

–

Other psychiatric % DSM-IV-TR ASD Sub-type

ASD autism spectrum disorder, DSM-IV-TR Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition-Text Revision, DSM-5 Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition, ADOS Autism Diagnostic Observation Schedule, ADHD attention deficit hyperactivity disorder, PDD-NOS pervasive developmental disorder-not otherwise specified * Fisher’s exact test was used thus there is no test statistic to report a

Dashed line (–) indicates that there were no subjects in this category

Table 3 Comparison between DSM-IV-TR and DSM-5 factor solutions using descriptive criteria and AIC/BIC indices TLI

RMSEA

BICa

SABICa

Chi square

Three-factor DSM-IV-TR

187.348*

.879

.850

.105

3,102.780

3,195.372

3,109.801

42.107*

.970

.958

.089

1,716.992

1,768.432

1,720.892

Two-factor DSM-5

CFI

AICa

Model tested

Smaller values of the AIC, BIC, and SABIC indices are indicative of better fit when comparing the two models DSM-IV-TR Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition-Text Revision, DSM-5 Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition, CFI Comparative Fit Index, TLI Tucker-Lewis Index, RMSEA root mean squared error of approximation, AIC Akaike Information Criterion, BIC Bayesian Information Criterion, SABIC Sample adjusted BIC * p \ .05 a

Raftery (1995) and Gignac and Watkins (2013) recommended that difference AIC and BIC values and their variants greater than 10 units reflect a large effect. Thus, using an effect size measure all model comparisons favored simpler models to a large extent (large effect size)

other words, IQ did not differentially affect either (a) the relative importance of each item to defining or operationalizing social reciprocity and repetitive behaviors, or (b) the mean level (threshold) of each item needed to determine clinical significance. Measurement Invariance by Age Two age groups were formed: B30 months and [30 months, consistent with groupings previously used in research studies (Barton et al. 2013; Guthrie et al. 2013).

123

The two factor solution was supported for both age groups, suggesting the equivalence of the two-factor configuration across ages, and all requirements of measurement invariance (configural, metric and scalar) were satisfied. However, the item thresholds of the younger group were consistently lower, meaning that fewer individual criteria for ASD were met. Due to the relatively small size of the younger group (N = 44), the relatively large standard errors may have masked the potential differences between age groups. Therefore, any conclusions we may draw from this finding can only be speculative at this time.

J Autism Dev Disord Table 4 Model fit when evaluating configural, metric and scalar invariance across gender, IQ, age, and DSM-IV-TR sub-types Type of invariance

Gender

IQ

Age

DSM-IV-TR sub-type ASD versus non-ASD

DSM-IV-TR sub-type ASD versus PDD/NOS versus non-ASD

X2, p value

v2(26) = 23.625, p = .597

v2(26) = 23.449, p = .607

v2(26) = 25.374, p = .498

v2(26) = 21.035, p = .740

v2(39) = 50.929, p = .096

RMSEA (C.I.); CFI

RMSEA = .000, C.I.s. = .000–.066; CFI = 1.00

RMSEA = .000, C.I.s. = .000–.068; CFI = 1.00

RMSEA = .000, C.I.s. = .000–.072; CFI = 1.00

RMSEA = .000, C.I.s. = .000–.055; CFI = 1.00

RMSEA = .065, C.I.s. = .000–.111; CFI = .985

X2, p value

v2(33) = 33.123, p = .461

v2(33) = 46.872, p = .055

v2(33) = 37.971, p = .253

_

_

RMSEA (C.I.); CFI

RMSEA = .006, C.Is. = .000–.069; CFI = 1.00

RMSEA = .063, C.Is. = .000–.102; CFI = .985

RMSEA = .036, C.Is. = .000–.081; CFI = .994

X2, p value

v2(36) = 41.218, p = .253

v2(36) = 45.032, p = .144

v2(36) = 33.528, p = .587

_

_

RMSEA (C.I.); CFI

RMSEA = .036, C.Is. = .000–.079; CFI = .994

RMSEA = .049, C.Is. = .000–.090; CFI = .990

RMSEA = .000, C.Is. = .000–.060; CFI = 1.00

Configural

Metric

Scalar

DSM-IV-TR Diagnostic and Statistical Manual of Mental Disorders-Fourth Edition-Text Revision, DSM-5 Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition, ASD autism spectrum disorder, RMSEA root mean squared error of approximation; CFI Comparative Fit Index Dashed line (–) indicates that this model could not be tested given that constant values were observed in some groups (e.g., all zeroes in the ASD group for a specific item)

Measurement Invariance by DSM-IV-TR sub-type To explore the question of differential exclusion of children with DSM-IV-TR diagnosed PDD-NOS and Asperger’s disorder under a DSM-5 model of ASD subjects were divided into ASD and non-ASD cases based on DSM-5 criteria. We then tested the probability of meeting DSM-5 ASD criteria based on DSM-IV-TR diagnostic status (autistic disorder, PDD-NOS, or No-ASD). Subjects with Asperger’s disorder were not included in this analysis due to small sample size (N = 5). With regard to the dichotomous grouping model (ASD versus non-ASD) the configural model fit the data properly. Metric and scalar invariance could not be tested, however, due to insufficient variance in certain key indicators between groups (e.g., all zeros in the ASD group for a specific criterion item); model solutions were therefore unavailable. To overcome this problem of non-convergence and to evaluate the hypothesis that the factor means were different in the two populations across both social communication and repetitive behavior, we employed a latent means model as part of a post hoc analysis. Results indicated a significant group effect, with the autistic disorder population having significantly higher means on both DSM-5 social communication (b = .108, p \ .001) and repetitive behaviors (b = .338, p \ .001) compared to the PDD-NOS population. This means that children with DSM-IV-TR autistic

disorder would be more likely to fulfill the DSM-5 social communication and repetitive behavior domains.

Discussion Our finding that the two-factor DSM-5 model for ASD is superior to the three-factor DSM-IV-TR is consistent with prior research (Frazier et al. 2008, 2012; Gotham et al. 2007; Guthrie et al. 2013; Mandy et al. 2012) and lends support to the implementation of the newly released DSM5 ASD criteria within clinical practices. Our findings indicate that children with PDD-NOS, and perhaps younger children, are less likely to meet DSM-5 ASD criteria. Perhaps the more psychometrically robust DSM-5 model is capturing the underlying biological phenomenon of ASD. In the absence of a biological marker for ASD the ability to analyze the psychometric properties of a behavioral diagnostic rubric is helpful although careful examination of which individuals do not meet the new DSM-5 criteria, and how to best support these individuals, is warranted. Consistent with our findings, several prior studies (Gibbs et al. 2012; Huerta et al. 2012; McPartland et al. 2012; Taheri and Perry 2012) have reported that individuals with PDD-NOS may be at risk for not meeting full DSM-5 ASD diagnostic criteria. While some prior studies have attempted

123

J Autism Dev Disord

to match existing data to the new DSM-5 criteria (McPartland et al. 2012; Taheri and Perry 2012), clinicians in our study concurrently collected data about both DSM-IV-TR and DSM-5 ASD criteria. A recently published paper (Young and Rodi 2013) used similar methodology with clinicians (psychologists and/or speech pathologists) assessing new patients referred to a large practice in South Australia. They also found that those with PDD-NOS were less likely to meet DSM-5 ASD criteria. There is great variation among clinical centers regarding diagnosis of ASD sub-types, such as PDD-NOS (Lord et al. 2012), thus our findings must be replicated in other centers. However, our data suggest that those with PDD-NOS are less likely to be diagnosed with ASD under the new DSM-5 criteria. This means that individuals who have social communication and behavioral impairments may not be recognized diagnostically and may not receive needed interventions to address their difficulties. Additionally, prior research has shown that children who do not meet full DSM-5 criteria for ASD may still have significant psychopathology known to co-occur with autism symptoms that could further contribute to poor outcomes if not recognized and treated (Rieske et al. 2013). The DSM-5 states that those with a well-established DSMIV-TR ASD diagnosis should not be re-evaluated (American Psychiatric Association 2013). However, it is not clear how individuals who would have met PDD-NOS criteria, but do not meet DSM-5 ASD criteria, should be managed once DSM-5 is operationalized. The diagnosis of PDD-NOS requires impairment in reciprocal social interaction associated with either impairment in communication skills or atypical behaviors (American Psychiatric Association 2000). Within DSM-5, individuals who have impairments in social communication but who lack restricted, repetitive behaviors or interests may be diagnosed with Social Communication Disorder. Kim et al. (2014) found that 32 % of children who meet DSM-IV-TR criteria for PDD-NOS may meet the criteria for Social Communication Disorder. However, given that Social Communication Disorder is newly recognized within DSM-5, it might not be recognized by insurers or schools, potentially limiting access to treatment for children with clinically significant impairment. Additionally, our findings suggest that these children have impairments in both social communication and repetitive, restricted behaviors, but at a sub-threshold level. These findings are consistent with prior research showing that many children who will no longer meet DSM criteria will still have significant symptoms of ASD (Worley and Matson 2012). Without an ASD diagnosis, children may not be able to access services that would have previously been granted to individuals with a PDDNOS diagnosis (Kulage et al. 2014). In our study, children younger than 30 months consistently met fewer DSM-5 ASD criteria. However, the relatively small number of young children (N = 44) precluded a full statistical

123

analysis. Given the emphasis on early diagnosis and intervention for ASDs (Maglione et al. 2012), the possibility that younger children may be less likely to meet DSM-5 diagnostic criteria is concerning. Prior studies have also found that young children may be less likely to meet DSM-5 ASD criteria, leading to speculation that relaxing the diagnostic criteria for young children may be necessary (Barton et al. 2013; Matson et al. 2012a; Matson et al. 2012b). Furthermore, it will be important to evaluate the developmental trajectory of these young children who manifest some degree of impairment in the social and communication domains. While our paper addresses diagnostic differences in ASD using the DSM-IV-TR versus DSM-5 criteria, these findings must be considered in the context of broader literature evaluating the validity of behavioral criteria to describe a biological phenomenon. An alternative interpretation of our findings is that the DSM-IV-TR model for ASD was too broad, especially the PDD-NOS sub-type, and may have led to the increasing prevalence of ASD reported in recent years. Confusion around how to best interpret our findings may come from difficulty resulting from classification of ASD on behavioral observations alone, as is done in the DSM system. The National Institute of Mental Health has voiced this concern and launched the Research Domain Criteria (RDoC) project to develop new ways of classifying psychopathology based on using dimensions of observable behavior coupled with neurobiological measures (Insel et al. 2010; Insel 2014). This perspective allows for pairing certain characteristics and studying them in tandem. An example might be determining whether recognizable neurobiological characteristics are associated with the atypical, repetitive behaviors often observed in individuals with ASD. There is evidence from a large study of over 3,000 twin pairs that the different traits of autism assessed in DSM-IVTR (social, communication, and behavioral impairments) are only modestly intercorrelated (Ronald et al. 2006). This suggests that evaluating certain behavioral characteristics of autism separately may be warranted (Happe et al. 2006). Further work elucidating the underlying neurobiological markers of autism is clearly needed in order to allow for better interpretation of the behavioral classification systems, and findings of our study must be considered with this in mind. Our findings must be interpreted in the context of some potential limitations. Our study was conducted at only one large academic medical center, thus limiting generalizability of the results. However, many different raters completed the DSM checklists and we found minimal interrater variability. Clinicians always completed the DSM-IVTR checklists prior to completion of the DSM-5 checklists, which may have influenced results. At the time of the checklists being completed, clinicians were still more knowledgeable about DSM-IV-TR because the DSM-5

J Autism Dev Disord

criteria had been newly proposed. Development of clinical tools that can help to elicit DSM-5 criteria may result in increased ASD diagnosis rates for those with seemingly sub-threshold symptoms. Despite these limitations, our study serves as one of the first to demonstrate both the psychometric strengths of the DSM-5 model and how the clinical application of DSM-5 criteria may affect which individuals are diagnosed with ASDs. Our study utilized concurrent collection of clinician completed DSM-IV-TR and DSM-5 checklists for ASD criteria for patients undergoing diagnostic assessment. Using confirmatory factor analysis, we found that the two-factor DSM-5 model is psychometrically superior to the threefactor DSM-IV-TR model. Within the DSM-5 model, children with PDD-NOS (under DSM-IV-TR) were less likely to receive an ASD diagnosis and young children tended to meet fewer DSM-5 criteria. Our findings suggest that a sub-group of children with impairments in social communication and/ or repetitive, restrictive behaviors may not meet criteria for a DSM-5 ASD diagnosis and thus may not receive services to address clinically significant impairment in these domains.



DSM  5ij ¼ b0j þ b1j  Teamij þ b2j  Ageij þ b3j

 Genderij þ b4j  DSM  4ij þ rij Level-2 Model b0j ¼ c00 þ u0j b1j ¼ c10 b2j ¼ c20 b3j ¼ c30 b4j ¼ c40 The prediction of DSM-5 scores is a function of the intercept b0j and the partial regression coefficients related to inter-rater team variability b1j participant’s age b2j gender b3j and DSM-IV-TR variability b4j plus the error of estimate rij. The classification variable at level-2 involved the number of different diagnostic teams. The presence of differential rater effects would be suggestive of bias in the criteria employed to clinically diagnose children with ASD. Thus, it represented an important prerequisite to validly testing the different classification systems.

Appendix 1 Multi-level model estimated to test the potential influence of inter-rater variability (bias). Level-1 Model

Appendix 2 See Table 5.

Table 5 Power analysis from simulating the responses of the two-factor model using IRT Estimatesa Population

Average

S.D.

S.E. average

M. S. E

95 % Cover

% Sig coeff

Social communication Item 1

1.000

1.0000

0.0000

0.0000

0.0000

1.0000

0.000

Item 2

0.850

0.8610

0.1541

0.1490

0.0238

0.960

1.000

Item 3

0.850

0.8610

0.1544

0.1491

0.0239

0.938

1.000

Repetitive behaviors Item 4

1.000

1.0000

0.0000

0.0000

0.0000

1.000

0.000

Item 5 Item 6

0.850 0.850

0.8658 0.8606

0.1347 0.1265

0.1248 0.1239

0.0184 0.0161

0.936 0.962

1.000 1.000

Item 7

0.850

0.8637

0.1250

0.1244

0.0158

0.950

1.000

The present Monte–Carlo simulation run with 500 replications suggesting that 200 participants were adequate to obtain both unbiased and statistically significant estimates of item slopes per factor Item Response Theory (IRT) reflects a family of models from contemporary psychometrics that are not based on the linear model. Specifically, the contribution of each item to a latent construct is evaluated based on the probability of answering it correctly (or endorsing it) termed threshold or item difficulty and the relationship between the item and the latent factor (slope) termed discrimination (Bond and Fox 2001; Hambleton and Swaminathan 1985; Rasch 1980; Reise 1990; Smith 2002; Smith et al. 1998) a

The first column describes the hypothesized slopes and the second the observed ones obtained from the simulation. The simulation was run with various seed numbers all of which produced identical results. The S.D. (3d column from left) is the standard deviation of the estimates across all 500 replications; The M.S.E. estimate reflects the amount of mean square error. The 95 % cover column (second from the right) shows the percentage of replications in which the population value was observed within the recommended 95 % confidence interval. The last column shows the percentage of replications for which the coefficients exceeded levels of significance at 5 %, thus it is an estimate of power. Estimates for the first item per factor are not available as they were fixed to unity for identification purposes. They do not represent improper estimates

123

J Autism Dev Disord

References Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723. Akaike, H. (1980). Likelihood and the Bayes procedure. In J. M. Bernardo (Ed.), Bayesian Statistics (Vol. 31, pp. 143–166). Valencia: University Press. American Psychiatric Association. (2000). Diagnostic and Statistical Manual of Mental Disorders-IV-Text Revision. Washington, DC: American Psychiatric Publishing. American Psychiatric Association. (2013). Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition. Arlington, VA: American Psychiatric Publishing. Amir, R. E., Van den Veyver, I. B., Wan, M., Tran, C. Q., Francke, U., & Zoghbi, H. Y. (1999). Rett syndrome is caused by mutations in X-linked MECP2, encoding methyl-CpG-binding protein 2. Nature Genetics, 23(2), 185–188. doi:10.1038/13810. Barton, M. L., Robins, D. L., Jashar, D., Brennan, L., & Fein, D. (2013). Sensitivity and specificity of proposed DSM-5 criteria for autism spectrum disorder in toddlers. Journal of Autism and Developmental Disorders, 43(5), 1184–1195. doi:10.1007/s10803-013-1817-8. Bayley, N. (2006). Manual for the Bayley Scales of Infant and Toddler Development (3rd ed.). San Antonio: The Psychological Corporation. Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606. Bollen, K. A. (1989). Structural equations with latent variables. New York, NY: Wiley. Bond, T. G., & Fox, C. M. (2001). Applying the Rasch model (2nd ed.). Mahwah, NJ: Lawrence Erlbaum. Breivik, E., & Olsson, U. (2001). Adding variables to improve fit: the effect of model size on fit assessment in Lisrel. In R. Cudeck, S. du Toit, & D. Sorbom (Eds.), Structural equation modeling: Present and future (pp. 169–194). Lincolnwood, IL: Scientific Software International. Brockwell, P. J., & Davis, R. A. (1991). Time series: theory and methods (2nd ed.). New York: Springer. Charan, S. H. (2012). Childhood disintegrative disorder. Journal of Pediatric Neurosciences, 7(1), 55–57. doi:10.4103/1817-1745. 97627. Elliott, C. (2007). Differential ability scales (2nd ed.). San Antonio: Pearson. Enders, C. K., & Tofighi, D. (2008). The impact of misspecifying class-specific residual variances in growth mixture models. Structural Equation Modeling: A Multidisciplinary Journal, 15, 75–95. Frazier, T. W., Youngstrom, E. A., Kubu, C. S., Sinclair, L., & Rezai, A. (2008). Exploratory and confirmatory factor analysis of the autism diagnostic interview-revised. Journal of Autism and Developmental Disorders, 38(3), 474–480. doi:10.1007/s10803007-0415-z. Frazier, T. W., Youngstrom, E. A., Speer, L., Law, P., Constantino, J., Findling, R. L., et al. (2012). Validation of proposed DSM-5 criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 28–40 e3. doi:10.1016/j.jaac.2011.09.021. Gibbs, V., Aldridge, F., Chandler, F., Witzlsperger, E., & Smith, K. (2012). Brief report: an exploratory study comparing diagnostic outcomes for autism spectrum disorders under DSM-IV-TR with the proposed DSM-5 revision. Journal of Autism and Developmental Disorders, 42(8), 1750–1756. doi:10.1007/s10803-0121560-6.

123

Gignac, G. E., & Watkins, M. W. (2013). Bifactor modeling and the estimation of model-based reliability in the WAIS-IV. Multivariate Behavioral Research, 48, 639–662. Gotham, K., Risi, S., Pickles, A., & Lord, C. (2007). The Autism Diagnostic Observation Schedule: revised algorithms for improved diagnostic validity. Journal of Autism and Developmental Disorders, 37(4), 613–627. doi:10.1007/s10803-006-0280-1. Guthrie, W., Swineford, L. B., Wetherby, A. M., & Lord, C. (2013). Comparison of DSM-IV and DSM-5 factor structure models for toddlers with autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 52(8), 797–805 e2. doi:10.1016/j.jaac.2013.05.004. Hambleton, R. K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer. Happe, F., Ronald, A., & Plomin, R. (2006). Time to give up on a single explanation for autism. Nature Neuroscience, 9(10), 1218–1220. doi:10.1038/nn1770. Hu, L. T., & Bentler, P. M. (1995). Evaluating model fit. In R. H. Hoyle (Ed.), Structural equation modeling concepts, issues, and applications (pp. 76–99). London: Sage. Hu, L. T., & Bentler, P. M. (1998). Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods, 3, 424–453. Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. Huerta, M., Bishop, S. L., Duncan, A., Hus, V., & Lord, C. (2012). Application of DSM-5 criteria for autism spectrum disorder to three samples of children with DSM-IV diagnoses of pervasive developmental disorders. The American Journal of Psychiatry, 169(10), 1056–1064. doi:10.1176/appi.ajp.2012.12020276. Hurvich, C. M., & Tsai, C.-L. (1989). Regression and time series model selection in small samples. Biometrika, 76(2), 297–307. Insel, T. R. (2014). The NIMH Research Domain Criteria (RDoC) Project: precision medicine for psychiatry. The American Journal of Psychiatry, 171(4), 395–397. doi:10.1176/appi.ajp. 2014.14020138. Insel, T., Cuthbert, B., Garvey, M., Heinssen, R., Pine, P. S., Quinn, K., et al. (2010). Research domain criteria (RDoC): Toward a new classification framework for research on mental disorders. The American Journal of Psychiatry, 167(7), 748–751. doi:10. 1176/appi.ajp.2010.09091379. Joreskog, K. (1973). A general method for estimating a linear structural equation system. In A. S. Goldberger & O. D. Duncan (Eds.), Structural equation models in the social sciences (pp. 85–112). New York: Seminar Press. Kenny, D. A. (2012). Measuring model fit. Retrieved October 22, 2012 from http://www.davidakenny.net/cm/fit.htm. Kent, R. G., Carrington, S. J., LeCouteur, A., Gould, J., Wing, L., Maljaars, J., et al. (2013). Diagnosing Autism Spectrum Disorder: who will get a DSM-5 diagnosis? Journal of Child Psychology and Psychiatry, 54(11), 1242–1250. doi:10.1111/jcpp.12085. Kim, Y. S., Fombonne, E., Koh, Y. J., Kim, S. J., Cheon, K. A., & Leventhal, B. L. (2014). A comparison of DSM-IV pervasive developmental disorder and DSM-5 autism spectrum disorder prevalence in an epidemiologic sample. Journal of the American Academy of Child and Adolescent Psychiatry, 53(5), 500–508. doi:10.1016/j.jaac.2013.12.021. Kulage, K. M., Smaldone, A. M., & Cohn, E. G. (2014). How Will DSM-5 affect autism diagnosis? A systematic literature review and meta-analysis. Journal of Autism and Developmental Disorders, 44(8), 1918–1932. doi:10.1007/s10803-014-2065-2. Lecavalier, L., Gadow, K. D., DeVincent, C. J., Houts, C., & Edwards, M. C. (2009). Deconstructing the PDD clinical phenotype: Internal validity of the DSM-IV. Journal of Child

J Autism Dev Disord Psychology and Psychiatry, 50(10), 1246–1254. doi:10.1111/j. 1469-7610.2009.02104.x. Loehlin, J. C. (2004). Latent variable models: An introduction to factor, path, and structural equation analysis. Mahwah, NJ: Lawrence. Lord, C., & Jones, R. M. (2012). Annual research review: Re-thinking the classification of autism spectrum disorders. Journal of Child Psychology and Psychiatry, 53(5), 490–509. doi:10.1111/j.14697610.2012.02547.x. Lord, C., Petkova, E., Hus, V., Gan, W., Lu, F., Martin, D. M., et al. (2012). A multisite study of the clinical diagnosis of different autism spectrum disorders. Archives of General Psychiatry, 69(3), 306–313. doi:10.1001/archgenpsychiatry.2011.148. Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavare, P. C., et al. (2000). The autism diagnostic observation schedule-generic: A standard measure of social and communication deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30(3), 205–223. MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149. Maglione, M. A., Gans, D., Das, L., Timbie, J., & Kasari, C. (2012). Nonmedical interventions for children with ASD: Recommended guidelines and further research needs. Pediatrics, 130(Suppl 2), S169–S178. doi:10.1542/peds.2012-0900O. Mahoney, W. J., Szatmari, P., Maclean, J. E., Bryson, S. E., Bartolucci, G., Walter, S. D., et al. (1998). Reliability and accuracy of differentiating pervasive developmental disorder subtypes. Journal of the American Academy of Child and Adolescent Psychiatry, 37(3), 278–285. doi:10.1097/00004583199803000-00012. Mandy, W. P., Charman, T., & Skuse, D. H. (2012). Testing the construct validity of proposed criteria for DSM-5 autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(1), 41–50. doi:10.1016/j.jaac. 2011.10.013. Marsh, H. W., Hau, K.-T., & Grayson, D. (2005). Goodness of fit in structural equation models. In A. Maydeu-Olivares & J. J. McArdle (Eds.), Contemporary psychometrics. A Festschrift for Roderick P. McDonald. Mahwah, NJ: Lawrence Erlbaum. Matson, J. L., Hattier, M. A., & Williams, L. W. (2012a). How does relaxing the algorithm for autism affect DSM-V prevalence rates? Journal of Autism and Developmental Disorders, 42(8), 1549–1556. doi:10.1007/s10803-012-1582-0. Matson, J. L., Kozlowski, A. M., Hattier, M. A., Horovitz, H., & Sipes, M. (2012b). DSM-IV vs DSM-5 diagnostic criteria for toddlers with autism. Developmental Neurorehabilitation, 15(3), 185–190. doi:10.3109/17518423.2012.672341. McPartland, J. C., Reichow, B., & Volkmar, F. R. (2012). Sensitivity and specificity of proposed DSM-5 diagnostic criteria for autism spectrum disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 51(4), 368–383. doi:10.1016/j.jaac. 2012.01.007. McQuarrie, A. D. R., & Tsai, C.-L. (1998). Regression and time series model selection. Singapore: World Scientific. Muthen, L. K., & Muthen, B. O. (2007). Mplus user’s guide 4. Los Angeles, CA: Muthen & Muthen. Norris, M., Lecavalier, L., & Edwards, M. C. (2012). The structure of autism symptoms as measured by the autism diagnostic observation schedule. Journal of Autism and Developmental Disorders, 42(6), 1075–1086. doi:10.1007/s10803-011-1348-0. Raftery, A. E. (1995). Bayesian model selection in social research. Sociological Methodology, 25, 111–163. Rasch, G. (1980). Probabilistic models for some intelligence and attainment tests. Chicago, IL: The University of Chicago Press.

Raykov, T. (2005). Studying group and time invariance in maximal reliability for multiple-component measuring instruments via covariance structure modelling. The British Journal of Mathematical and Statistical Psychology, 58(Pt 2), 301–317. doi:10. 1348/000711005X38591. Raykov, T., & Marcoulides, G. (2000). A first course in structural equation modeling. Mahwah, NJ: Lawrence. Reise, S. (1990). A comparison of item and person fit methods of assessing model fit in IRT. Applied Psychological Measurement, 42, 127–137. Rieske, R. D., Matson, J. L., Beighley, J. S., Cervantes, P. E., Goldin, R. L., & Jang, J. (2013). Comorbid psychopathology rates in children diagnosed with autism spectrum disorders according to the DSM-IV-TR and the proposed DSM-5. Developmental Neurorehabilitation: Advance online publication. doi:10.3109/ 17518423.2013.790519. Rigdon, E. E. (1996). CFI versus RMSEA: A comparison of two fit indexes for structural equation modeling. Structural Equation Modeling, 3(4), 369–379. Ronald, A., Happe, F., Bolton, P., Butcher, M., Price, T. S., & Plomin, R. (2006). Genetic heterogeneity between the three components of the autism spectrum: A twin study. Journal of the American Academy of Child and Adolescent Psychiatry, 45(6), 691–699. doi:10.1097/01.chi.0000215325.13058.9d. Schwarz, G. E. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461–464. Sideridis, G. D., Simos, P., Papanicolaou, A., & Fletcher, J. (2014). Using structural equation modeling to assess functional connectivity in the brain: Power and sample size considerations. Educational and Psychological Measurement, 74, 733–758. Smith, E. V, Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3, 205–231. Smith, R. M., Schumacker, R. E., & Bush, M. J. (1998). Using item mean squares to evaluate fit to the Rasch model. Journal of Outcome Measurement, 2, 66–78. Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25(2), 173–180. Steiger, J. H. (2000). Point Estimation, hypothesis testing, and interval estimation using the RMSEA: Some comments and a reply to Hayduk and Glaser. Structural Equation Modeling, 7(2), 149–162. Steiger, J. H. (2007). Understanding the limitations of global fit assessment in structural equation modeling. Personality and Individual Differences, 42, 893–898. Stuive, I., Kiers, H. A. L., Timmerman, M. E., & ten Berge, J. M. F. (2008). The empirical verification of an assignment of items to subtests: The oblique multiple group method versus the confirmatory common factor method. Educational and Psychological Measurement, 68(6), 923–939. Taheri, A., & Perry, A. (2012). Exploring the proposed DSM-5 criteria in a clinical sample. Journal of Autism and Developmental Disorders, 42(9), 1810–1817. doi:10.1007/s10803-0121599-4. Tofghi, D., & Enders, C. K. (2007). Identifying the correct number of classes in mixture models. In G. R. Hancock & K. M. Samulelsen (Eds.), Advances in latent variable mixture models (pp. 317–341). Greenwich, CT: Information Age. Tucker, L. R., & Lewis, C. (1973). The reliability coefficient for maximum likelihood factor analysis. Psychometrica, 38, 1–10. Widaman, K. F., & Thompson, J. S. (2003). On specifying the null model for incremental fit indices in structural equation modeling. Psychological Methods, 8(1), 16–37.

123

J Autism Dev Disord Worley, J. A., & Matson, J. L. (2012). Comparing symptoms of autism spectrum disorders using the current DSM-IV-TR diagnostic criteria and the proposed DSM-V diagnostic criteria. Research in Autism Spectrum Disorders, 6, 965–970.

123

Young, R. L., & Rodi, M. L. (2013). Redefining Autism Spectrum Disorder Using DSM-5: The Implications of the Proposed DSM-5 Criteria for Autism Spectrum Disorders. Journal of Autism and Developmental Disorders, 44(4), 758–765. doi:10.1007/s10803-013-1927-3.