PsychologicalReports, 1990, 67, 55-63. O Psychological Reports 1990
RELATIONSHIP BETWEEN SCORES O N DEVELOPING COGNITIVE ABILITIES TEST AND SCORES O N MEDICAL COLLEGE ADMISSIONS TEST FOR NONTRADITIONAL PREMEDICAL STUDENTS ' PAUL HENRY AND HAROLD R. BARDO
Medical Education (MEDPREP), Soufhern Illinois University, School of Medicine
Summary.-The relationship of students' performance on the Developing Cognitive Abilities Test (DCAT), a test of scholastic aptitude, and their subsequent performance on the Medical College Admission Test (MCAT) were examined for 122 nontraditional premedical students who participated in a medical educational preparatory program. A stepwise multiple regression analysis produced moderate, though significant multiple correlations among- subscores on the two tests. While there were a few exceptions, for the most part all of the subscores on the Developing Cognitive Abilities Test made a significant contribution to the regression equation in the prediction of scores on MCAT subtests. Implications for the value of the Developing Cognitive Abilities Test as an admissions tool as well as providing direction for possible internention are discussed.
Despite a recent decline in the size of the pool of applicants, admissions to medical school have continued to be competitive (Gordon, 1978; McCrae, 1978). Students' performance on the Medical College Admission Test (MCAT) ranked high among the preadmission variables and was usualIy given considerable weight by medical schools in deciding which application would be given the highest consideration for admission (Lloyd & Miller, 1989). Also, some medical schools took into consideration the number of times the applicant took the MCAT by assigning negative weights to the application of multiple MCAT takers. Students have strived to do well init i d y on the test. Considering the above, predicting performance on the MCAT seems to be an important aspect of the admissions process. The power of the MCAT in predicting success in medical school has been demonstrated (Golrnan & Berry, 1981; McGuite, 1982; Jones, 1983). An earlier test that successfully predicted MCAT performance could also, by extension, help identify those students possessing academic potential for acceptance into medical school. For example, Thurmond and Lewis (1986) reported that scores on the Scholastic Aptitude Test (SAT) correlated moderately with MCAT scores. O n the basis of their findings, these authors suggested that students who are likely to perform well on the MCAT can be identified while they are still undergraduates. The present study was undertaken to examine the relationship among the six subrests of the MCAT and 'Address correspondence to Paul Henry, Southern Illinois University, School of Medicine, Carbondale, IL 62901-4323.
56
P. HENRY & H. R. BARD0
the eight subtests of the Developing Cognitive Abilities Test (Beggs & Mouw, 1980) to ascertain the predictive validity of the DCAT.
Participants A total of 122 students (43 men and 79 women), ranging in age from 20 to 34 yr. (with a mean age of 22 yr.) participated in this study. These participants were either undergraduate students (i.e., juniors or seniors) or postgraduate students enrolled in a medical education preparatory program at a midwestern university. The program was designed to increase the number of applicants who are underrepresented in the medical profession (i.e., racial minorities and rural whites). Given the nature of the preparatory program, a wide diversity of individuals are represented both in the program and in this sample. These individuals differ in regard to academic background and socioeconomic status. The program was designed to provide an opportunity for individuals who might not otherwise enter the medical profession given past academic histories or other circumstances. Two general types of students are enrolled: (1) reentering students, both postgraduate and undergraduate, who have been working and wish to change career fields and (2) continuing students, both postgraduate and undergraduate, who have been unsuccessful in their attempts to matriculate into medical school or who are applying to medical school for the first time. The present sample of 122 individuals consisted of 93 postgraduates and 27 undergraduates (12 juniors and 17 seniors). The mean grade point average (GPA) for this sample was 2.73 on a scale of 1.0 to 4.0.
Program Activities This program has offered special tutorials, seminars, and classes to the enrolled students. The curriculum included (a) courses to correct deficiencies in science, quantitative and verbal skills, testmanship, time management and study skills, (b) developmental courses which were designed to supplement or review premedical science courses in biology, (c) extensive tutorial assistance under program supervision for campus courses in biology, chemistry, physics, and mathematics, (d) counseling and advisement, (e) career development seminars which included an orientation to the application process, preparation for the professional interview, and activities designed to promote the student's self-awareness, and (f) exposure to portions of the first-year professional school curriculum. Students must have completed an application, submitted transcripts and letters of recommendations to be invited to an Admissions Day program. At the admissions program applicants are interviewed and given a battery of tests, including the Developing Cognitive Abilities Test (Beggs & Mouw, 1980). Students who have demonstrated potential, based on submitted information, to become competitive applicants to
NONTRADITIONAL ADMISSIONS TO MEDICAL COLLEGE
57
medical school, test and interview results, and show a need for program services are generally admitted by the MEDPREP Admissions Committee.
Testing Instruments The independent variable was the Developing Cognitive Abilities Test which participants took as part of a battery of MEDPREP admissions tests. The test measured the characteristics and abihties that contribute to academic performance. Unlike traditional mental ability test, the test was based on the assumption that characteristics and abilities can be altered and improved through instruction. Two dimensions of aptitude were assessed; the first, and more traditional, dimension included verbal, quantitative, and spatial abilities, and the second dimension provided information base on five of six cognitive levels of Bloom's (1956) taxonomy: (a) knowledge, (b) comprehension, (c) application, (d) analysis, and (e) synthesis. Test items (80 items) were constructed to represent both content areas and cognitive levels. Scores are reported by subtest in both the content area and the cognitive taxonomy as well as at a total score level. The test yields eight subtest scores and a total score. The subtests produced reliability coefficients (KR-20) ranging from .49 (Synthesis) to '86 (Verbal). Intercorrelations among the subtests ranged from .35 to .80, with a median coefficient of .42. Evidence of the predictive valiBty of the tests, i.e., predicting achievement test scores, has been described elsewhere (Henry & Bardo, 1987). The dependent variable or criterion measure was the Medical College Admissions Test (MCAT). Participants took the MCAT after being enrolled in the MEDPREP program for two or three semesters. The MCAT is a standardized, multiple-choice examination required by almost all U.S. medical schools. Six scores are reported on a scale of 1 to 15, with 1 being the lowest and 15 the highest score. Evidence of vahdity and reliabdity of the MCAT has been established (McGuire, 1982; Jones, 1983).
Research Question and Analysis The major question addressed by the present study was how well can scores on the MCAT be predicted from scores on the Developing Cognitive Abilities Test (content areas and cognitive taxonomy) taken 9 months to one year earlier. To address the research question a stepwise multiple regression analysis was computed for each dimension of the Developing Cognitive Abilities Test.
RESULTS The means, standard deviations and minimum and maximum score values are presented in Table 1. The results presented in Table 2 indicate that all four subscores on the Developing Cognitive Abihties Test content dimen~ sion (Verbal, Quantitative, Spatial and Total Score) add significantly ( p .05)
58
P. HENRY & H. R. BARD0
to the regression equation in the prediction of all MCAT subtests (Biology, Chemistry, Physics, Science Problems, Reading, and Quantitative). TABLE 1 MEANSCORES,STANDARD DEVIATIONS, AND RANGES FORSTUDENTS'PERFORMANCE ON COG^ ABILITIES TEST MEDICALCOLLEGEADMISSIONS TESTAND DEVELOPING Variable Developing Cognitive Abilities Test Verbal Dimension Quantitative Dimension Spatial Dimension Total Test Score Knowledge Cornprehension Application Analysis Synthesis Medical CoUege Admissions Test Biology Chemistry Physics Science Problems Reading Ouantitative
M
SD
22.50 20.97 13.26 56.79 13.23 12.44 12.73 10.14 8.29
2.68 3.81 4.33 8.25 1.37 2.12 2.58 2.82 2.46
7.86 6.55 5.70 5.41 5.48 4.23
1.61 2.68 1.86 1.81 1.84 2.17
Range 14.00-26.00 12.00-28.00 6.00-21.00 37.00-74.00 10.00-15.00 7.00-16.00 6.00-17.00 6.00-17.00 4.00-14.00 5.00-12.00 3.00-11.00 3.00-11.00 3.00-11.00 3.00-10.00 1.00- 7.00
The four aptitude subscores together account for 34%, 3O%, 33%, 49%, 48%, and 42% of the variance in the MCAT Biology, Chemistry, Physics, Science Problems, Reading and Quantitative subtests, respectively. From Table 2 it is also apparent that DCAT Total score was the best single predictor (on the DCAT Content Dimension) since it entered the model first in the prediction of four of the six MCAT subtests (Biology, Physics, Science Problems, and Quantitative). Chemistry and Reading were the two MCAT subtests in which DCAT Total did not enter the regression equation first. DCAT Quantitative was the best single predictor of MCAT Chemistry (accounting for 27% of the variance) while DCAT Verbal was the best single predictor of MCAT Reading (accounting for 30% of the variance). In the prediction of MCAT Biology as the first entered variable in the stepwise analysis DCAT Total accounted for 31% ( p < .01) of the variance (when the variance associated with the other Developing Cognitive Ability Test scores were controlled). When DCAT Quantitative and DCAT Spatial were added to the equation, respectively, each variable, though significant, accounted for less than 1% of the increment in the explained variance ( R 2 ) , DCAT Verbal, the final variable to enter the equation, accounted for a 2% increment in the explained variance (R2) to make a total of 34% ( P C.05) of the variance in MCAT Biology scores.
59
NONTRADITIONAL ADMISSIONS TO MEDICAL COLLEGE
In the prediction of MCAT Chemistry, DCAT Qualitative (the best single predictor) accounted for 27% (p
RELATIONSHIP BETWEEN SCORES O N DEVELOPING COGNITIVE ABILITIES TEST AND SCORES O N MEDICAL COLLEGE ADMISSIONS TEST FOR NONTRADITIONAL PREMEDICAL STUDENTS ' PAUL HENRY AND HAROLD R. BARDO
Medical Education (MEDPREP), Soufhern Illinois University, School of Medicine
Summary.-The relationship of students' performance on the Developing Cognitive Abilities Test (DCAT), a test of scholastic aptitude, and their subsequent performance on the Medical College Admission Test (MCAT) were examined for 122 nontraditional premedical students who participated in a medical educational preparatory program. A stepwise multiple regression analysis produced moderate, though significant multiple correlations among- subscores on the two tests. While there were a few exceptions, for the most part all of the subscores on the Developing Cognitive Abilities Test made a significant contribution to the regression equation in the prediction of scores on MCAT subtests. Implications for the value of the Developing Cognitive Abilities Test as an admissions tool as well as providing direction for possible internention are discussed.
Despite a recent decline in the size of the pool of applicants, admissions to medical school have continued to be competitive (Gordon, 1978; McCrae, 1978). Students' performance on the Medical College Admission Test (MCAT) ranked high among the preadmission variables and was usualIy given considerable weight by medical schools in deciding which application would be given the highest consideration for admission (Lloyd & Miller, 1989). Also, some medical schools took into consideration the number of times the applicant took the MCAT by assigning negative weights to the application of multiple MCAT takers. Students have strived to do well init i d y on the test. Considering the above, predicting performance on the MCAT seems to be an important aspect of the admissions process. The power of the MCAT in predicting success in medical school has been demonstrated (Golrnan & Berry, 1981; McGuite, 1982; Jones, 1983). An earlier test that successfully predicted MCAT performance could also, by extension, help identify those students possessing academic potential for acceptance into medical school. For example, Thurmond and Lewis (1986) reported that scores on the Scholastic Aptitude Test (SAT) correlated moderately with MCAT scores. O n the basis of their findings, these authors suggested that students who are likely to perform well on the MCAT can be identified while they are still undergraduates. The present study was undertaken to examine the relationship among the six subrests of the MCAT and 'Address correspondence to Paul Henry, Southern Illinois University, School of Medicine, Carbondale, IL 62901-4323.
56
P. HENRY & H. R. BARD0
the eight subtests of the Developing Cognitive Abilities Test (Beggs & Mouw, 1980) to ascertain the predictive validity of the DCAT.
Participants A total of 122 students (43 men and 79 women), ranging in age from 20 to 34 yr. (with a mean age of 22 yr.) participated in this study. These participants were either undergraduate students (i.e., juniors or seniors) or postgraduate students enrolled in a medical education preparatory program at a midwestern university. The program was designed to increase the number of applicants who are underrepresented in the medical profession (i.e., racial minorities and rural whites). Given the nature of the preparatory program, a wide diversity of individuals are represented both in the program and in this sample. These individuals differ in regard to academic background and socioeconomic status. The program was designed to provide an opportunity for individuals who might not otherwise enter the medical profession given past academic histories or other circumstances. Two general types of students are enrolled: (1) reentering students, both postgraduate and undergraduate, who have been working and wish to change career fields and (2) continuing students, both postgraduate and undergraduate, who have been unsuccessful in their attempts to matriculate into medical school or who are applying to medical school for the first time. The present sample of 122 individuals consisted of 93 postgraduates and 27 undergraduates (12 juniors and 17 seniors). The mean grade point average (GPA) for this sample was 2.73 on a scale of 1.0 to 4.0.
Program Activities This program has offered special tutorials, seminars, and classes to the enrolled students. The curriculum included (a) courses to correct deficiencies in science, quantitative and verbal skills, testmanship, time management and study skills, (b) developmental courses which were designed to supplement or review premedical science courses in biology, (c) extensive tutorial assistance under program supervision for campus courses in biology, chemistry, physics, and mathematics, (d) counseling and advisement, (e) career development seminars which included an orientation to the application process, preparation for the professional interview, and activities designed to promote the student's self-awareness, and (f) exposure to portions of the first-year professional school curriculum. Students must have completed an application, submitted transcripts and letters of recommendations to be invited to an Admissions Day program. At the admissions program applicants are interviewed and given a battery of tests, including the Developing Cognitive Abilities Test (Beggs & Mouw, 1980). Students who have demonstrated potential, based on submitted information, to become competitive applicants to
NONTRADITIONAL ADMISSIONS TO MEDICAL COLLEGE
57
medical school, test and interview results, and show a need for program services are generally admitted by the MEDPREP Admissions Committee.
Testing Instruments The independent variable was the Developing Cognitive Abilities Test which participants took as part of a battery of MEDPREP admissions tests. The test measured the characteristics and abihties that contribute to academic performance. Unlike traditional mental ability test, the test was based on the assumption that characteristics and abilities can be altered and improved through instruction. Two dimensions of aptitude were assessed; the first, and more traditional, dimension included verbal, quantitative, and spatial abilities, and the second dimension provided information base on five of six cognitive levels of Bloom's (1956) taxonomy: (a) knowledge, (b) comprehension, (c) application, (d) analysis, and (e) synthesis. Test items (80 items) were constructed to represent both content areas and cognitive levels. Scores are reported by subtest in both the content area and the cognitive taxonomy as well as at a total score level. The test yields eight subtest scores and a total score. The subtests produced reliability coefficients (KR-20) ranging from .49 (Synthesis) to '86 (Verbal). Intercorrelations among the subtests ranged from .35 to .80, with a median coefficient of .42. Evidence of the predictive valiBty of the tests, i.e., predicting achievement test scores, has been described elsewhere (Henry & Bardo, 1987). The dependent variable or criterion measure was the Medical College Admissions Test (MCAT). Participants took the MCAT after being enrolled in the MEDPREP program for two or three semesters. The MCAT is a standardized, multiple-choice examination required by almost all U.S. medical schools. Six scores are reported on a scale of 1 to 15, with 1 being the lowest and 15 the highest score. Evidence of vahdity and reliabdity of the MCAT has been established (McGuire, 1982; Jones, 1983).
Research Question and Analysis The major question addressed by the present study was how well can scores on the MCAT be predicted from scores on the Developing Cognitive Abilities Test (content areas and cognitive taxonomy) taken 9 months to one year earlier. To address the research question a stepwise multiple regression analysis was computed for each dimension of the Developing Cognitive Abilities Test.
RESULTS The means, standard deviations and minimum and maximum score values are presented in Table 1. The results presented in Table 2 indicate that all four subscores on the Developing Cognitive Abihties Test content dimen~ sion (Verbal, Quantitative, Spatial and Total Score) add significantly ( p .05)
58
P. HENRY & H. R. BARD0
to the regression equation in the prediction of all MCAT subtests (Biology, Chemistry, Physics, Science Problems, Reading, and Quantitative). TABLE 1 MEANSCORES,STANDARD DEVIATIONS, AND RANGES FORSTUDENTS'PERFORMANCE ON COG^ ABILITIES TEST MEDICALCOLLEGEADMISSIONS TESTAND DEVELOPING Variable Developing Cognitive Abilities Test Verbal Dimension Quantitative Dimension Spatial Dimension Total Test Score Knowledge Cornprehension Application Analysis Synthesis Medical CoUege Admissions Test Biology Chemistry Physics Science Problems Reading Ouantitative
M
SD
22.50 20.97 13.26 56.79 13.23 12.44 12.73 10.14 8.29
2.68 3.81 4.33 8.25 1.37 2.12 2.58 2.82 2.46
7.86 6.55 5.70 5.41 5.48 4.23
1.61 2.68 1.86 1.81 1.84 2.17
Range 14.00-26.00 12.00-28.00 6.00-21.00 37.00-74.00 10.00-15.00 7.00-16.00 6.00-17.00 6.00-17.00 4.00-14.00 5.00-12.00 3.00-11.00 3.00-11.00 3.00-11.00 3.00-10.00 1.00- 7.00
The four aptitude subscores together account for 34%, 3O%, 33%, 49%, 48%, and 42% of the variance in the MCAT Biology, Chemistry, Physics, Science Problems, Reading and Quantitative subtests, respectively. From Table 2 it is also apparent that DCAT Total score was the best single predictor (on the DCAT Content Dimension) since it entered the model first in the prediction of four of the six MCAT subtests (Biology, Physics, Science Problems, and Quantitative). Chemistry and Reading were the two MCAT subtests in which DCAT Total did not enter the regression equation first. DCAT Quantitative was the best single predictor of MCAT Chemistry (accounting for 27% of the variance) while DCAT Verbal was the best single predictor of MCAT Reading (accounting for 30% of the variance). In the prediction of MCAT Biology as the first entered variable in the stepwise analysis DCAT Total accounted for 31% ( p < .01) of the variance (when the variance associated with the other Developing Cognitive Ability Test scores were controlled). When DCAT Quantitative and DCAT Spatial were added to the equation, respectively, each variable, though significant, accounted for less than 1% of the increment in the explained variance ( R 2 ) , DCAT Verbal, the final variable to enter the equation, accounted for a 2% increment in the explained variance (R2) to make a total of 34% ( P C.05) of the variance in MCAT Biology scores.
59
NONTRADITIONAL ADMISSIONS TO MEDICAL COLLEGE
In the prediction of MCAT Chemistry, DCAT Qualitative (the best single predictor) accounted for 27% (p
