European Journal of Dental Education ISSN 1396-5883

Changes in students’ perceptions of their dental education environment I. Kang1, L. A. Foster Page2, V. R. Anderson3, W. M. Thomson4 and J. M. Broadbent2 1 2 3 4

Faculty of Dentistry, Sir John Walsh Research Institute, University of Otago, Dunedin, New Zealand, Department of Oral Rehabilitation, Faculty of Dentistry, Sir John Walsh Research Institute, University of Otago, Dunedin, New Zealand, College of Education, University of Otago, Dunedin, New Zealand, Department of Oral Sciences, Faculty of Dentistry, Sir John Walsh Research Institute, University of Otago, Dunedin, New Zealand

keywords dental educational environment; Dundee Ready Educational Environment Measure; Bachelor of Dental Surgery; students’ perceptions; questionnaire; curriculum. Correspondence Lyndie A. Foster Page Department of Oral Rehabilitation Faculty of Dentistry Sir John Walsh Research Institute University of Otago PO Box 647 Dunedin 9054, New Zealand Tel: 034795853 Fax: 034795079 e-mail: [email protected] Accepted: 19 May 2014 doi: 10.1111/eje.12112

Abstract Objectives: To examine the ‘Expected’ and ‘Actual’ educational environment experienced by a cohort of Bachelor of Dental Surgery (BDS) students at the University of Otago’s Faculty of Dentistry using the Dundee Ready Educational Environment Measure (DREEM). Methods: Cohort members were asked to complete five DREEM surveys over the four-year BDS programme. Student expectations at the beginning of their first year were assessed using a modified version of the DREEM questionnaire, while following (standard) DREEM questionnaires at the end of each professional year addressed students’ ‘Actual’ perception of the educational environment. Results: Sixty-six students (99%) completed at least one questionnaire. Overall, the BDS students’ perceptions of their educational environment tended to be positive and students identified both perceived strengths and weaknesses in the BDS programme. However, more negative than positive shifts were reported between the ‘Expected’ and ‘Actual’ individual DREEM individual items, suggesting that BDS students initially expected more from their educational environment than they actually experienced. Individual DREEM outcomes undergoing negative and positive shifts differed over the years and varied in number. These may be explained, in part, by changes in the curriculum focus from year to year. Conclusion: The students’ changing DREEM responses over time revealed anticipated and perceived strengths and weaknesses of the BDS curriculum, as well as shifts in students’ perceptions in response to curricular changes. However, our findings highlight the potential usefulness for dental education of a measure for use that takes the unique aspects of the dental education environment into account.

Introduction Within professional programmes, the educational environment and curriculum can be seen as closely intertwined. In the past, the definition of curriculum referred to traditional didactic teaching (1). However, more recent literature has focused on students’ broader educational environments, suggesting that everything that happens within an educational context constitutes ‘curriculum’ (2–5). 122

Students’ perceptions of their health professional educational environments have been increasingly studied over the past decades. This is because the learning environment shapes ‘curriculum’, is a significant determinant of behaviour and contributes to students’ academic achievement, course satisfaction and emerging professional aspirations (2, 6, 7). Such understanding has led to the development of numerous measures by health professional educators for analysis of students’ perspectives of their actual curricula (5). Arguably, a mismatch between students’ ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

Kang et al.

expectations and experiences of a health professional programme may lead to a higher chance of dissatisfaction and attrition. Tracking students’ experiences of their educational programmes over time may highlight areas where staff development and adjustments to the formal curriculum are needed (8). The Dundee Ready Educational Environment Measure (DREEM), the inventory of choice for this study, has been determined to be the most suitable instrument to be applied in undergraduate medical learning environments (9). The DREEM was developed using a form of grounded theory with a panel of nearly 100 medical educators and 1000 students to measure and ‘diagnose’ undergraduate educational climates in the health professions (10). The 50-item closed-question questionnaire aims to identify students’ perceptions of five aspects of the educational environment: perceptions of learning, teaching, academic self-perceptions, perceptions of atmosphere and social self-perceptions. The DREEM has been used to: identify the strengths and weaknesses of educational environments; compare different medical education institutions; compare students at different stages of a course; and evaluate differences by gender and educational background (11–15). Dental educators have been slower to develop measures to examine students’ learning environments and experiences, although some investigators have studied effective clinical teaching (16–19), with one measure developed and piloted in 2005 (19). The most recent development in this field is the Dental Clinical Learning Environment Instrument (DECLEI), which was developed and validated in Athens using contemporary psychometric standards, and is used for measuring undergraduate dental students’ perceptions of the clinical educational environment (20). In dental education, the content validity of many of the instruments modified from those developed and validated in the medical educational setting has been questioned (9). In comparison with its broad international adaptation in medical education settings (10, 11), the DREEM has now been applied in crosssectional studies at a small number of dental faculties, including Greece (12), India (13), Germany (11), Pakistan (14), and New Zealand (15), and it has been shown to have validity in the dental education environment. However, no published research has, as yet, used the DREEM to follow a cohort of students through their entire dental educational programme or tracked their perceptions of their dental educational environment over time. This study aimed to examine how a cohort of Bachelor of Dental Surgery (BDS) students perceived their educational environment during their entire programme and to determine how their expectations and experiences of the educational environment changed over time.

Method The student cohort for this study included those who were in their first professional year of the BDS programme at the Faculty of Dentistry (at the University of Otago, New Zealand) in 2009. The University of Otago is the sole provider of the BDS programme in New Zealand, providing a 5-year degree programme that begins with a competitive first-year Health Sciences course. After that, students are selected to commence the BDS course in their second academic year; alternatively, a ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

Students’ perceptions of their dental education environment

small number of students may be accepted into the programme if they have finished the second year of another degree or completed a degree/diploma. Hence, what is referred to in the remainder of this study as the second (BDS2) academic year is equivalent to students’ first professional year. The first three professional years of the programme (BDS2, BDS3 and BDS4) involve mainly didactic and laboratory teaching, with an increase in clinical teaching from the end of BDS2 onwards. The final year is primarily a clinical year integrated with outplacements in community dental facilities across New Zealand. The 50-item DREEM scores each item on a five-point Likert scale, where 4 = strongly agree, 3 = agree, 2 = unsure, 1 = disagree and 0 = strongly disagree. It includes nine negative statements, which are subsequently reverse coded, so that a higher score represents a more positive perception. The 50 statements can be further divided into five subscales: (i) perception of learning (12 items); (ii) perception of teachers (11 items); (iii) academic self-perceptions (8 items); (iv) perception of atmosphere (12 items); and (v) social self-perceptions (7 items) (10). Prior to the commencement of this study, ethical approval was obtained from the University of Otago Human Ethics Committee. Five questionnaires were then administered over 4 years. The first survey was conducted in the first week of the first professional year (2009) to assess students’ expectations of their new educational environment. A version of the DREEM asking students to think about what they expected their first year to be like was formed by altering the wording of the DREEM items. This questionnaire was labelled as the ‘Expected’ DREEM. For example, the item ‘I am encouraged to participate in class’ was changed to ‘I will be encouraged to participate in class’. These items were preceded by: ‘You are just starting your first year at the Faculty of Dentistry, University of Otago, and we would like to ask you some questions about how you think it will be. Please think about what you expect this year to be like and, with this in mind, indicate whether you agree or disagree with each of the following statements’. The second, third, fourth and fifth surveys were conducted at the ends of the first, second, third and fourth professional years (respectively) to determine students’ perceptions of the educational climate experienced throughout the BDS programme. These questionnaires were labelled as the ‘Actual’ DREEM. Socio-demographic characteristics (gender and age) were collected in the first questionnaire. Each time the questionnaire was delivered, students were informed that it was part of a research project and not part of their academic programme, that their participation was voluntary and that their ID numbers would be used for tracking purposes only. A senior dental student administered the questionnaires. As this was a longitudinal cohort study being conducted over 4 years, there were concerns about respondent burden, so no investigation of test/ retest reliability was undertaken. Statistical analysis was undertaken using SPSS (version 21.0; SPSS IBM, Armonk, NY, USA). The 50 items were summed to give the total DREEM score (out of 200) for the 2009 ‘Expected’ and ‘Actual’ surveys, 2010 ‘Actual’, 2011 ‘Actual’ and 2012 ‘Actual’. Scores for the five subscales and individual DREEM items were also summed and mean scores calculated. The total and subscale DREEM scores were interpreted using descriptors developed by McAleer and Roff (21). The guideline 123

Students’ perceptions of their dental education environment

also defined score groups for the 50 individual DREEM items, with mean scores ≥3.5 indicating strengths in the educational environment, and scores ≤2.0 indicating weak areas. The Wilcoxon signed rank test for related samples was used for comparison between the ‘Actual’ and ‘Expected’ DREEM scores at both individual item and subscale levels. To analyse the statistical significance of changes in students’ perceptions of their educational environments between survey periods, the Wilcoxon signed rank test for related samples was conducted with a 0.01 significance level. A Bonferroni-adjusted alpha level was used due to the large number of statistical tests. The non-parametric Mann–Whitney U-test and Kruskal–Wallis test were used to test the statistical significance of observed sociodemographic differences by gender and age, respectively. Effectsize statistics were calculated by dividing the mean of the change scores by the standard deviation of the baseline scores (comparing the 2009 ‘Expected’ score to the 2012 ‘Actual’) to give a dimensionless measure of effect. Effect-size statistics of 0.7 a large change. Group-based trajectory analysis of DREEM scores (and subscales) was conducted in Intercooled Stata (version 10.0) (22). The Bayesian information criterion (BIC) was used as the criterion for model selection. Data analysis included every participant who responded to the questionnaire at least once. Tests were conducted for any impact on trajectories by age and gender.

Results The sample comprised 67 students who joined the BDS programme at the beginning of 2009 and graduated in 2012; of those, 66 (99%) completed at least one DREEM questionnaire and 44 (66%) completed all five questionnaires. Response rates for each stage of the study ranged from 82% (for the 2009 ‘Actual DREEM’) to 94% (for the 2010 ‘Actual’ DREEM). Equal numbers of male and female students participated in the study (33 each). The mean age of the students was 23 years (range 21–37). Summary data on the overall DREEM and subscale scores from each stage are presented in Table 1 and Fig. 1. The overall DREEM scores reflected more positive than negative perceptions, being 145 of 200 for 2009 ‘Expected’ to 143, 132, 134 and 134 for the 2009, 2010, 2011 and 2012 ‘Actual’ DREEM, respectively. Significant negative shifts were noted between the ‘Expected’ and the three ‘Actual’ overall DREEM scores (P < 0.01) at the end of second, third and fourth professional years. The only significant decrease in ‘Actual’ overall DREEM scores was between the first and second professional years (P < 0.01). All subscale scores remained in the ‘above average’ category. The ‘Expected’ subscale DREEM scores for ‘Learning’ and ‘Academic’ were consistently (and significantly) greater than the respective subscale scores of the four ‘Actual’ DREEM surveys. The significant decreases in ‘Actual’ subscale DREEM scores were found only between the first and second professional years, in the subscale results for learning, teachers and atmosphere. There was no positively significant shift reported in the overall and subscale data. 124

Kang et al.

Table 1 displays the mean scores for each individual DREEM item from all five surveys. The strengths and weaknesses evident in BDS students’ responses to each survey, as identified by the score descriptors (with scores ≥3.5 suggesting positive aspects, and scores ≤2.0 suggesting problematic aspects), are presented in Table 2. At the beginning of the first professional year, BDS students predicted three strengths and two areas of weakness. Three individual DREEM items were repeatedly identified as areas of weakness in all of the following stages of the ‘Actual’ DREEM with the biggest number of weaknesses reported at the end of the third professional year. Statistically significant shifts in the ‘Expected’ and ‘Actual’ individual DREEM data are also presented in Table 1. The BDS students indicated significant positive changes in four individual DREEM items, relative to their initial ‘Expected’ DREEM at the end of the first year. One individual item showed consistent significant positive shifts over all four years (that ‘the atmosphere is relaxed during lectures’). The number of significant negative shifts between the ‘Expected’ and ‘Actual’ individual DREEM items was notably greater than the number of positive shifts. Students’ responses revealed a total of ten significant negative changes in individual DREEM items by the end of the first professional year with this number rising to 20 by the end of the fourth professional year. There were six items where the students reported consistent negative deviations from their initial expectation over 4 years. These items are shown in Fig. 2 along with effect sizes. All individual DREEM items had large effect sizes except for the item ‘the teaching is focused’, which had a moderate effect size of 0.7. A large effect size was found for the subscale perception of learning (0.9), with moderate effect sizes occurring for the subscales perception of teachers, academic self-perceptions and perception of atmosphere (0.6, 0.6 and 0.4, respectively), while the social self-perceptions subscale had the smallest effect size of 0, reflecting no change. Group-based trajectory analysis of overall and subscale DREEM scores across time identified two distinct DREEM trajectory groups (which we designated ‘low’ and ‘high’). Their mean overall and subscale DREEM trajectories are plotted in Fig. 3. There were no systematic differences by gender and age between those in the high and low trajectory groups (data not presented here). For the overall DREEM, the high trajectory group decreased from 148.5 (SD 14.9) at the start to 139.9 (SD 12.5) at the end of the BDS programme; on the other hand, the low trajectory group decreased from 136.8 (SD 13.7) to 117.8 (SD 16.0) during the same period. On average, the low trajectory group started the course with a lower mean score and had a greater decrease in mean scores through the course.

Discussion To our knowledge, this is the first study to take a longitudinal approach with the validated DREEM inventory and follow a cohort of students through their entire dental education programme. This investigation aimed to provide greater understanding of what happens within the entire BDS course provided by the Faculty of Dentistry at the University of Otago, for an objective analysis of the curriculum. The students’ ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

Kang et al.

Students’ perceptions of their dental education environment

TABLE 1. Mean subscale, individual and overall DREEM scores for the 2009, 2010, 2011 and 2012 surveys

Expected DREEM Overall DREEM Students’ perceptions of learning I will be encouraged to participate in class The teaching will often be stimulating The teaching will be student-centred The teaching will help to develop my competence The teaching will be well focused The teaching will help to develop my confidence The teaching time will be put to good use The teaching will over-emphasise factual learning1 I will be clear about the learning objectives of the course The teaching will encourage me to be an active learner Long-term learning will be emphasised over short-term learning The teaching will be too teachercentred1 Students’ perceptions of teachers The teachers will be knowledgeable The teachers will be patient with students The teachers will ridicule the students1 The teachers will be authoritarian1 The teachers will have good communication skills with students The teachers will be good at providing feedback to students The teachers will provide constructive criticism The teachers will give clear examples The teachers will get angry in class1 The teachers will be well prepared for their classes The students will irritate the teachers1 Students’ academic self-perceptions Learning strategies which worked for me before will continue to work for me now I am confident about passing this year I feel I will be well prepared for my profession Last year’s work will have been a good preparation for this year’s work I will be able to memorise all I need I will learn a lot about empathy in my profession

Actual DREEM

I am encouraged to participate in class The teaching is often stimulating The teaching is student-centred The teaching helps to develop my competence The teaching is well focused The teaching helps to develop my confidence The teaching time is put to good use The teaching over-emphasises factual learning1 I am clear about the learning objectives of the course The teaching encourages me to be an active learner Long-term learning is emphasised over short-term learning The teaching is too teachercentred1 The teachers are knowledgeable The teachers are patient with students The teachers ridicule the students1 The teachers are authoritarian1 The teachers have good communication skills with students The teachers are good at providing feedback to students The teachers provide constructive criticism The teachers give clear examples The teachers get angry in class1 The teachers are well prepared for their classes The students irritate the teachers1 Learning strategies which worked for me before continue to work for me now I am confident about passing this year I feel I am being well prepared for my profession Last year’s work has been a good preparation for this year’s work I am able to memorise all I need I have learned a lot about empathy in my profession

ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

2009 ‘Expected’ Mean (SD)

2009 ‘Actual’ Mean (SD)

2010 ‘Actual’ Mean (SD)

2011 ‘Actual’ Mean (SD)

2012 ‘Actual’ Mean (SD)

145 (15.4) 36 (4.4) 3.5 (0.5)

143 (16.8) 34 (5.1)* 3.2 (0.6)*

132 (17.0)*,† 31 (5.4)*,† 2.7 (0.8)*,†

134 (15.0)* 31 (6.3)* 2.9 (0.7)*

134 (16.5)* 32 (6.1)* 2.6 (0.8)*

3.3 (0.7) 3.0 (0.7) 3.6 (0.6)

3.0 (0.7) 2.9 (0.8) 3.4 (0.7)

2.8 (0.7)* 2.6 (0.7)* 3.1 (0.7)*

2.7 (0.9)* 2.7 (0.8)* 3.0 (0.8)*

2.8 (0.7)* 2.7 (0.8)* 3.0 (0.7)*

3.2 (0.7) 3.3 (0.6)

2.9 (0.9)* 3.0 (0.9)

2.7 (0.6)* 2.8 (0.8)*

2.7 (0.9)* 2.6 (0.9)*

2.7 (0.8)* 2.8 (0.8)*

3.3 (0.7)

2.8 (0.8)*

2.5 (0.7)*

2.4 (0.8)*

2.3 (1.0)*

1.8 (0.6)

1.8 (1.0)

2.1 (1.0)

1.8 (0.9)

2.3 (0.8)*,§

2.9 (0.8)

2.4 (0.9)*

2.2 (0.9)*

2.5 (0.8)*

2.4 (0.9)*

3.2 (0.7)

2.9 (0.8)

2.7 (0.8)*

2.7 (0.8)*

2.8 (0.8)*

3.1 (0.7)

2.8 (0.9)

2.6 (0.8)*

2.7 (1.0)

2.7 (1.0)

2.3 (0.7)

2.7 (0.8)*

2.5 (0.8)

2.4 (0.8)

2.4 (0.7)

32 (4.7) 3.7 (0.5) 3.0 (0.8)

33 (4.5) 3.7 (0.5) 3.1 (0.7)

30 (4.2)† 3.3 (0.6)† 2.7 (0.7)†

28 (4.3)* 3.3 (0.6)* 2.6 (0.8)*

29 (5.2)* 3.3 (0.6)* 2.8 (0.7)

2.9 (0.9)

2.9 (1.0)

2.5 (0.9)

2.4 (1.0)*

2.6 (0.9)

2.0 (0.8) 3.1 (0.7)

2.4 (1.1) 3.1 (0.7)

2.0 (1.0)† 2.8 (0.7)†

2.1 (1.0) 2.7 (0.6)*

2.1 (0.9) 2.8 (0.6)*

2.8 (0.8)

3.0 (0.7)

2.6 (0.9)

2.0 (1.0)*,‡

1.9 (1.0)*

3.0 (0.7)

3.1 (0.8)

2.9 (0.6)

2.6 (0.7)*

2.5 (0.8)*

3.1 (0.7) 2.8 (0.8) 3.3 (0.6)

2.9 (0.8)* 3.1 (1.0)* 3.2 (0.6)

2.8 (0.6) 2.9 (1.0) 2.9 (0.6)*,†

2.9 (0.6) 2.6 (1.1) 3.0 (0.6)*

2.7 (0.8)* 2.7 (0.9) 2.8 (0.8)*

2.5 (0.9)

2.8 (0.9)

2.4 (0.9)

2.2 (0.9)

2.4 (0.9)

24 (3.4) 2.7 (0.8)

21 (3.9)* 2.7 (0.9)

21 (3.2)* 2.5 (0.8)

22 (3.2)* 2.5 (0.9)

22 (3.4)* 2.4 (1.0)

3.1 (0.6)

2.8 (0.8)

3.2 (0.5)†

3.1 (0.5)

3.2 (0.6)

3.1 (0.6)

2.8 (0.8)*

2.6 (0.8)*

2.7 (0.7)*

2.5 (0.8)*

3.1 (0.7)

2.2 (1.1)*

2.7 (0.6)*,†

2.8 (0.8)

3.0 (0.6)

2.2 (0.9) 3.2 (0.6)

1.9 (1.1) 2.9 (1.0)

1.9 (1.0) 2.8 (0.8)*

1.9 (1.0) 3.0 (0.6)

2.0 (1.0) 3.0 (0.6)

125

Students’ perceptions of their dental education environment

Kang et al.

Table 1. Continued

Expected DREEM My problem-solving skills will be well developed here Much of what I have to learn will seem relevant to a career in healthcare Students’ perceptions of atmosphere The atmosphere will be relaxed during the clinic teaching This school will be well timetabled Cheating will be a problem in this school1 The atmosphere will be relaxed during lectures There will be opportunities for me to develop interpersonal skills I will feel comfortable in class socially The atmosphere will be relaxed during seminars/tutorials I will find the experience disappointing1 I will be able to concentrate well The enjoyment will outweigh the stress of studying at the School of Dentistry The atmosphere will motivate me as a learner I will feel able to ask the questions I want Students’ social self-perceptions There will be a good support system for students who get stressed I will be too tired to enjoy this course1 I will be rarely bored on this course I will have good friends in this school My social life will be good I will seldom feel lonely My accommodation will be pleasant

Actual DREEM My problem-solving skills are being well developed here Much of what I have to learn seems relevant to a career in healthcare The atmosphere is relaxed during the clinic teaching This school is well timetabled Cheating is a problem in this school1 The atmosphere is relaxed during lectures There are opportunities for me to develop interpersonal skills I feel comfortable in class socially The atmosphere is relaxed during seminars/tutorials I find the experience disappointing1 I am able to concentrate well The enjoyment outweighs the stress of studying at the School of Dentistry The atmosphere motivates me as a learner I feel able to ask the questions I want There is a good support system for students who get stressed I am too tired to enjoy this course1 I am rarely bored on this course I have good friends in this school My social life is good I seldom feel lonely My accommodation is pleasant

2009 ‘Expected’ Mean (SD)

2009 ‘Actual’ Mean (SD)

2010 ‘Actual’ Mean (SD)

2011 ‘Actual’ Mean (SD)

2012 ‘Actual’ Mean (SD)

3.3 (0.6)

2.9 (0.8)*

2.8 (0.7)*

3.1 (0.6)‡

3.0 (0.7)*

3.4 (0.6)

3.2 (0.9)

3.0 (0.8)*

3.1 (0.6)

3.1 (0.7)*

33 (4.6) 2.3 (0.8)

35 (4.8) 2.9 (0.9)*

31 (4.9)† 2.1 (1.1)†

31 (4.7) 2.2 (1.0)

32 (4.4) 2.3 (1.0)

2.3 (1.1) 2.3 (1.4)

1.9 (1.2) 2.4 (1.3)

1.5 (1.1)* 2.1 (1.3)

1.1 (1.1)*,‡ 1.7 (1.2)*,‡

1.0 (0.9)* 2.1 (1.2)§

2.3 (0.8)

3.1 (0.7)*

2.9 (0.6)*

2.9 (0.6)*

3.0 (0.6)*

3.2 (0.5)

3.2 (0.6)

2.9 (0.7)*

3.2 (0.5)‡

3.1 (0.5)

3.2 (0.6) 2.5 (0.8)

3.4 (0.6) 3.2 (0.6)*

3.1 (0.6) 2.9 (0.7)

3.3 (0.6) 3.1 (0.6)*

3.2 (0.5) 3.1 (0.5)*

3.2 (0.8)

3.2 (1.0)

3.0 (0.8)

3.0 (0.9)

3.0 (0.8)

2.9 (0.6) 2.9 (0.8)

2.6 (0.8) 2.9 (1.0)

2.7 (0.7) 2.6 (0.9)

2.7 (0.7) 2.7 (0.9)

2.7 (0.7) 2.8 (0.8)

3.0 (0.8)

2.9 (0.9)

2.7 (0.7)

2.7 (0.8)

2.8 (0.8)

3.0 (0.7)

3.1 (0.7)

2.7 (0.9)†

2.7 (0.8)

2.9 (0.7)

20 (2.9) 2.9 (0.6)

20 (3.6) 2.0 (0.8)*

18 (3.5)* 1.9 (1.0)*

20 (3.1) 2.0 (0.8)*

20 (3.6) 2.0 (0.9)*

2.6 (0.7)

2.4 (1.0)

2.2 (1.0)

2.4 (1.0)

2.6 (0.8)

2.5 3.3 3.0 2.7 3.2

2.4 3.6 3.3 3.1 3.0

2.3 3.4 3.1 2.6 2.9

2.4 3.5 3.2 3.0 3.3

2.6 3.5 3.3 2.9 3.1

(1.0) (0.7) (0.6) (0.9) (0.6)

(1.1) (0.7) (0.8) (0.9) (1.0)

(1.0) (0.6) (0.7) (1.0)† (1.0)

(1.0) (0.6) (0.7) (1.0) (0.7)

(0.9) (0.6) (0.6)* (1.0) (1.0)

SD, standard deviation. The nine items in italics show the negative statements that have been reverse-coded (higher score means more positive evaluation). * Significant difference from the start of the first professional year (P < 0.01, Wilcoxon signed rank test). † Significant difference between the end of the first and second professional year (P < 0.01, Wilcoxon signed rank test). ‡ Significant difference between the end of the second and third professional year (P < 0.01, Wilcoxon signed rank test). § Significant difference between the end of the third and fourth professional year (P < 0.01, Wilcoxon signed rank test). 1

responses over time revealed ‘Expected’ (2009) and ‘Actual’ (2009, 2010, 2011, 2012) strengths and weaknesses of the BDS educational environment. While four other studies have reported findings using the DREEM in dental education contexts (11–14), their study designs were cross-sectional, and compared groups of students in different professional years. The use of trajectory analysis for investigating latent groups in DREEM score changes over time in the current study is novel, and lends insight into the changes within different groups of

126

students. Thus, it is difficult to compare our study findings with those previously reported. The DREEM was initially established for the ‘development and validation of a universal inventory that equips health professions/medical educators with a diagnostic tool to measure the state of their school’s learning and teaching climate’ (10). Using this inventory, curriculum designers could consider investigating the individual DREEM items that showed consistent negative deviation from the students’ initial expectation.

ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

Students’ perceptions of their dental education environment

% of maximum score

Kang et al.

Subscale DREEM Fig. 1. Mean DREEM scores as a percentage of maximum scores for the 2009, 2010, 2011 and 2012 surveys.

TABLE 2. Predicted and actual strengths and weaknesses in the educational environment for the 2009, 2010, 2011 and 2012 surveys

2009 ‘Expected’ DREEM

2009 ‘Actual’ DREEM

Strengths (mean individual DREEM score ≥3.5)

Weaknesses (mean individual DREEM score ≤2.0)

I will be encouraged to participate in class The teaching will help to develop my competence The teachers will be knowledgeable The teachers are knowledgeable I have good friends in this school

The teaching will over-emphasise factual learning1 The teachers will be authoritarian1

2010 ‘Actual’ DREEM

None

2011 ‘Actual’ DREEM

I have good friends in this school

2012 ‘Actual’ DREEM

I have good friends in this school

The teaching over-emphasises factual learning1 I am able to memorise all I need This school is well timetabled There is a good support system for students who get stressed The teachers will be authoritarian1 I am able to memorise all I need This school is well timetabled There is a good support system for students who get stressed The teaching over-emphasises factual learning1 The teachers are good at providing feedback to students I am able to memorise all I need This school is well timetabled Cheating is a problem in this school1 There is a good support system for students who get stressed The teachers are good at providing feedback to students I am able to memorise all I need This school is well timetabled There is a good support system for students who get stressed

1

The items in italics show the negative statements that have been reverse-coded.

For example, based on students’ negative responses to the DREEM item ‘I feel I am being well prepared for my profession’, curriculum designers could invite students to define ‘preparedness’; and to identify the aspects of dentistry for which they feel unprepared, the factors that might affect their confidence in the identified aspects, and the teaching approaches that are most likely to enhance students’ confidence. In this study, the students’ ‘Expected’ and ‘Actual’ perceptions of their overall educational environment were more positive than negative. In all five of the DREEM surveys, the overall DREEM scores were in the middle to higher end of the ‘above average’ category. The four overall ‘Actual’ DREEM scores were higher than all of the published overall DREEM scores from other evaluated dental education settings (11–14), suggesting that the New Zealand dental students had more a positive overª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

all perception of their dental education environment than did those in the other studies. Nevertheless, the New Zealand students’ perceptions still ranked their educational environment as being substantially lower than a supposedly ‘excellent’ educational environment. The repeated deviations between the students’ overall ‘Expected’ and ‘Actual’ DREEM scores indicated that the dental curriculum failed to meet students’ expectations in some areas. The students initially had a high expectation of their new educational environment. This may have been an outcome of their participation in an extremely competitive admission process prior to entering the course. Interestingly, however, the students did not expect an excellent environment (which would be indicated by a score of 151–200). In our earlier validation study, we suggested that students’ prior knowledge of the 127

Kang et al.

Individual DREEM score

Students’ perceptions of their dental education environment

Fig. 2. Individual DREEM items which had significantly negative shifts from the ‘Expected’ to the ‘Actual’ DREEM scores (effect sizes shown above bar).

40

Learning subscale

Subscale score 25 30 35

140 130 120

20

110

Scale score

150

Overall DREEM scale

2

10

22 Months of study Low (28.0%)

34

2

10

High (72.0%)

Teachers subscale 28

Perceptions subscale Subscale score 22 24 26

35 30

18

20

25

Subscale score

20 2

10

2

22 34 Months of study Low (31.2%) High (68.8%)

10

34 High (45.7%)

Social subscale

16

25

30

35

Subscale score 18 20 22

24

40

22 Months of study Low (54.3%)

Atmosphere subscale

Subscale score

22 34 Months of study Low (22.7%) High (77.3%)

2

10

22 Months of study Low (44.3%)

34 High (55.7%)

2

10

22 34 Months of study Low (54.1%) High (45.9%)

Fig. 3. Trajectory plot of mean overall and subscale DREEM scores for a two group trajectory analysis model.

128

ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

Kang et al.

Faculty’s older facilities and/or their conversations with senior dental students who were already in the programme may have resulted in their ‘high’ (rather than ‘excellent’) ‘Expected’ DREEM scores (15). Group-based trajectory modelling enabled profiling of variation in changes in DREEM scores over time within the cohort. Two group models were specified for the overall DREEM scores and subscales. We previously hypothesised that, ‘if all students had realistic (rather than overly optimistic) expectations, there may be less chance of dissatisfaction and attrition’ (15). However, conversely, the outcome of the trajectory analysis suggests that the students who began the course with a higher expectation of their learning environment reported, having more positive perceptions of the environment overall through the rest of the course (with the exception of their expectation and perception of teachers, in which the two trajectory groups started with very similar expectation levels). Future research could examine the role of students’ general outlook in their perceptions of the educational environment. Perhaps students’ perceptions are shaped not only by their educational experiences, but also by their general outlook or disposition, and whether this is more positive or negative (22). Curriculum changes and shifts in curricular emphases over the 4 years could account for the statistically significant shifts in students’ overall, subscale and individual DREEM outcomes. The biggest changes in individual and subscale ‘Actual’ DREEM outcomes were noted between the first and second professional years. This could be explained by the commencement of clinical dentistry, involving contact with patients. Clinical dentistry with patients begins in the second professional year. The significant decrease in the ‘learning’, ‘teachers’ and ‘atmosphere’ subscales (as well as in the nine individual DREEM scores) could be due to the students feeling more pressured in this new clinical teaching environment. Similar negative differences in subscale DREEM scores between students in pre-clinical and clinical parts of the curriculum were also reported in the DREEM results of a Greek dental school (12). The reduction in the number of negative shifts between the ‘Actual’ individual DREEM scores over the following years could be an indication of the students’ adaptation to their clinical setting and the pressures associated with increased clinical hours. Changes that could also be attributed to the gradual increase in the clinical component of the curriculum were the significant positive changes related to the students’ confidence in their knowledge. Similar positive differences in students’ academic self-perceptions between pre-clinical and clinical aspects of dental curriculum were also reported in Greek and German studies (11, 12). Curriculum content, organisation and teacher attitude seemed to form the common basis of the majority of the reported weaknesses. Again, the simplistic models of teacher effectiveness that attribute learning outcomes to the actions of the tutor and ignore the web of other influences may be unfair to teachers (23), however, clear organisation for didactic instruction has been identified as the most important qualities of an effective teacher in dentistry (24). Interestingly, the weaknesses perceived by New Zealand dental students were not dissimilar to those identified by students in Greek and Indian dental faculties (12, 13). Curriculum content and organisation are two aspects of an environment within a Faculty that can be ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

Students’ perceptions of their dental education environment

changed at an institutional level. However, teacher attitude has been shown to be more difficult and may require a culture change (18). A limitation of the DREEM for dental educational environments is that it does not include questions specifically able to recognise differences in educational aspects of the dental programme. For example, the clinical setting used in an undergraduate dental education environment is very different to a university lecture or seminar room and even to undergraduate medical training. Unlike medical students, dental students in their second professional year are undertaking operative procedures on patients under tutor supervision, although this differs among dental schools. Although this study did not set out to measure clinical teaching effectiveness, we hypothesise that many of the shifts are due to this. The skills and techniques that students develop during their clinical education are heavily dependent on their interactions with the faculty who serve as mentors and evaluators (25). Studies in the 1980s and 1990s investigating clinical teaching effectiveness and the clinical learning environment and more recently studies understanding students’ perceptions of clinical teaching identified many aspects of ‘what effective clinical teachers do’ (20, 25). Instruments that have been developed to measure students’ experiences of clinical teaching environments include the Postgraduate Hospital Educational Environment Measure (PHEEM), which was developed for use in clinical teaching contexts in medicine using a similar methodology as the DREEM, the effective clinical dental teaching (ECDT) scale, and the DECLEI which has been developed, validated and piloted for use in dental education specifically (19, 20, 26). These findings reflect on a New Zealand BDS programme and offer productive insights into the issues that need to be addressed to develop a desirable educational environment for future dental practitioners in New Zealand. They also raise interesting questions for future dental education research. For example, future research could examine: whether similar changes in students’ perceptions of their ‘expected’ and ‘actual’ dental education environments are replicated elsewhere; whether the use of an instrument that measures clinical teaching environments specifically reveals similar trajectories; and the role of personality or ‘outlook’ in shaping students’ experiences and/or perceptions of dental education. More in-depth qualitative investigations of the dental education environment could enhance our understanding of both the strengths and weaknesses of the current context, and of wider factors shaping students’ perceptions of their dental educational environments.

Conclusion The students’ changing DREEM responses over time revealed anticipated and perceived strengths and weaknesses of the BDS curriculum, as well as shifts in students’ perceptions in response to curricular changes. This study’s aim was not to consider clinical teaching in isolation, but to explore students’ overall experience of their educational environment over time. However, our findings highlight the potential usefulness of a measure that can take into account the unique aspects of the dental education environment. 129

Students’ perceptions of their dental education environment

Kang et al.

Funding New Zealand Dental Association/Division of Health Sciences Summer Research Studentship.

13

14

References 1 Harden RM. The learning environment and the curriculum. Med Teach 2001: 23: 335–336. 2 Genn JM. AMEE Medical Education Guide No. 23 (Part 1): curriculum, environment, climate, quality and change in medical education – a unifying perspective. Med Teach 2001: 23: 337–344. 3 Masella RS. The hidden curriculum: value added in dental education. J Dent Educ 2006: 70: 279–283. 4 Tan AS, Anderson VR, Foster Page LA. Second and third year oral health and dental student perceptions of future professional work. Eur J Dent Educ 2013: 17: 241–250. 5 Karnieli-Miller O, Vu TR, Holtman MC, Clyman SG, Inui TS. Medical students’ professionalism narratives: a window on the informal and hidden curriculum. Acad Med 2010: 85: 124–133. 6 Lizzio A, Wilson K, Simons R. University students’ perceptions of the learning environment and academic outcomes: implications for theory and practice. Stud Higher Educ 2002: 27: 27–52. 7 Miles S, Leinster SJ. Medical students’ perceptions of their educational environment: ‘Expected’ versus ‘Actual’ perceptions. Med Educ 2007: 41: 265–272. 8 Foster Page LA, Kang M, Anderson VR, Thomson WM. Appraisal of the Dundee Ready Educational Environment Measure in the New Zealand dental education environment. Eur J Dent Educ 2012: 16: 78–85. 9 Soemantri D, Herrera C, Riquelme A. Measuring the educational environment in health professions studies: a systemic review. Med Teach 2010: 32: 947–952. 10 Roff S, McAleer S, Harden RM, et al. Development and validation of the Dundee Ready Education Environment Measure (DREEM). Med Teach 1997: 19: 295–299. 11 Ostapczuk MS, Hugger A, de Bruin J, Ritz-Timme S, Rotthoff T. DREEM on, dentists! Students’ perceptions of the educational environment in a German dental school as measured by the Dundee Ready Education Environment Measure. Eur J Dent Educ 2012: 16: 67–77. 12 Kossioni AE, Varela R, Ekonomu I, Lyrakos G, Dimoliatis DK. Students’ perceptions of the educational environment in a Greek

130

15

16 17 18

19

20

21

22 23

24

25

26

Dental School, as measured by DREEM. Eur J Dent Educ 2011: 15: 1–6. Thomas BS, Abraham RR, Alexander M, Ramnarayan K. Students’ perceptions regarding educational environment in an Indian dental school. Med Teach 2009: 31: e185–e188. Ali K, Raja M, Phil M, Watson G, Coombes L, Heffernan E. The dental school learning milieu: students’ perceptions at five academic dental institutions in Pakistan. J Dent Educ 2012: 76: 487–494. Foster Page LA, Kang M, Anderson V, Thomson WM. Appraisal of the Dundee Ready Educational Environment Measure in the New Zealand dental educational environment. Eur J Dent Educ 2011: 15: 1–8. Myers B. Beliefs of dental faculty and students about clinical teaching behaviours. J Dent Educ 1977: 41: 68–76. Emling RC, Fritz LS. Dental student and faculty perceptions of teacher characteristics. J Dent Educ 1978: 42: 78–82. Manague M, Brown G, Foster H. Clinical assessment of dental students: values and practices of teachers in restorative dentistry. Med Educ 2001: 35: 364–370. McGrath C, Wai R, Comfort MB, McMillian AS. Development and evaluation of a questionnaire to evaluate initial dental teachers (ECDT). Br Dent J 2005: 198: 45–48. Kossioni AE, Lyrakos G, Ntinalexi I, Varela R, Economu I. The development and validation of a questionnaire to measure the clinical learning environment for undergraduate dental students (DECLEI). Eur J Dent Educ 2014: 18: 71–79. Roff S, McAleer S, Harden R, et al. Development and validation of the Dundee ready education environment measure (DREEM). Med Teach 1997: 1: 295–299. Jones BL, Nagin DS. A note on a Stata plugin for estimating groupbased trajectory models. Soc Methods Res 2013: 42: 608–613. Postlethwaite K, Maull W. Understanding students’ changing perceptions of their learning environments in four FE colleges in England. Educ Rev 2007: 59: 429–449. Schonwetter B, Lavigne S, Mazurat R, Nazarko O. Students’ perceptions of effective classroom and clinical teaching in dental and dental hygiene education. J Dent Educ 2006: 70: 624–635. Henzi D, Davis E, Jasinevicius R, Hendricson W. North American dental students’ perspectives about their clinical education. J Dent Educ 2006: 70: 361–377. Roff S, McAleer S, Skinner A. Development and validation of an instrument to measure the postgraduate clinical learning and teaching educational environment for hospital-based junior doctors in the UK. Med Teach 2005: 27: 326–331.

ª 2014 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd Eur J Dent Educ 19 (2015) 122–130

Copyright of European Journal of Dental Education is the property of Wiley-Blackwell and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

Changes in students' perceptions of their dental education environment.

To examine the 'Expected' and 'Actual' educational environment experienced by a cohort of Bachelor of Dental Surgery (BDS) students at the University ...
327KB Sizes 0 Downloads 5 Views