Memory

ISSN: 0965-8211 (Print) 1464-0686 (Online) Journal homepage: http://www.tandfonline.com/loi/pmem20

No pain no gain: The positive impact of punishment on the strategic regulation of accuracy Michelle M. Arnold, Lisa M. Chisholm & Toby Prike To cite this article: Michelle M. Arnold, Lisa M. Chisholm & Toby Prike (2014): No pain no gain: The positive impact of punishment on the strategic regulation of accuracy, Memory, DOI: 10.1080/09658211.2014.990982 To link to this article: http://dx.doi.org/10.1080/09658211.2014.990982

Published online: 20 Dec 2014.

Submit your article to this journal

Article views: 110

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at http://www.tandfonline.com/action/journalInformation?journalCode=pmem20 Download by: [University of Lethbridge]

Date: 11 November 2015, At: 11:27

Memory, 2014 http://dx.doi.org/10.1080/09658211.2014.990982

No pain no gain: The positive impact of punishment on the strategic regulation of accuracy Michelle M. Arnold, Lisa M. Chisholm, and Toby Prike

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

School of Psychology, Flinders University, Adelaide, SA, Australia

(Received 29 May 2014; accepted 17 November 2014)

Previous studies have shown that punishing people through a large penalty for volunteering incorrect information typically leads them to withhold more information (metacognitive response bias), but it does not appear to influence their ability to distinguish between their own correct and incorrect answers (metacognitive accuracy discrimination). The goal of the current study was to demonstrate that punishing people for volunteering incorrect information—versus rewarding volunteering correct information— produces more effective metacognitive accuracy discrimination. All participants completed three different general-knowledge tests: a reward test (high points for correct volunteered answers), a baseline test (equal points/penalties for volunteered correct/incorrect answers) and a punishment test (high penalty for incorrect volunteered answers). Participants were significantly better at distinguishing between their own correct and incorrect answers on the punishment than reward test, which has implications for situations requiring effective accuracy monitoring.

Keywords: Strategic regulation; Punishment; Reward; Metacognition; Type-2 signal detection theory.

Metacognitive research has become increasingly mainstream within the memory and cognition domain, and one important branch is the strategic regulation of accuracy; that is, how well people use the knowledge they have about their own cognitive processes and memories to guide their performance. Several studies have demonstrated that changing metacognitive response bias (i.e., your willingness to report a response) can be relatively easy. However, metacognitive accuracy discrimination—your ability to distinguish between your own correct and incorrect responses—has shown to be much more difficult to manipulate (Arnold, Higham, & Martín-Luengo, 2013; Higham & Arnold, 2007; Koriat & Goldsmith, 1996). For

example, instituting a large versus small penalty for volunteering incorrect information typically leads to a more conservative response bias (withholding more answers) but does not significantly impact people’s ability to distinguish between their correct and incorrect answers (e.g., Higham, 2007; Koriat & Goldsmith, 1994). The goal of the current study was to demonstrate that, when directly compared, a large pay-off for reporting a correct answer (reward) leads people to be worse at discriminating between their own correct and incorrect responses than when they receive a large loss for reporting an incorrect answer (punishment). One reason it is not clear what impact, if any, gains versus losses have on our ability to

Address correspondence to: Michelle M. Arnold, School of Psychology, Flinders University, GPO Box 2100, Adelaide, SA 5001, Australia. E-mail: [email protected] The authors thank Ben Maddock for technical assistance, Ines Jentzsch and Paul Williamson for initial discussions about the results, and Asher Koriat and an anonymous reviewer for their comments on an earlier draft of the manuscript. Portions of this research were presented at the 54th Annual Meeting of the Psychonomic Society, Toronto, ON, Canada.

© 2014 Taylor & Francis

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

2

ARNOLD, CHISHOLM, PRIKE

accurately discriminate our own information is because researchers typically manipulate only one type of cost/benefit outcome; for instance, high versus low reward (Higham, 2007; Koriat & Goldsmith 1994, 1996; Roebers & Fernandez, 2002; Roebers, Moga, & Schneider, 2001; Roebers & Schneider, 2005). For example, Higham (2007) found that participants who were given a large penalty (–4) for reporting incorrect information had a more conservative response bias than participants given a small penalty (–.25), but that penalty size had no influence on accuracy monitoring: Participants in both groups performed similarly when it came to distinguishing between their own correct and incorrect answers. We argue that, to see differences in metacognitive discrimination, both rewards and punishments need to be manipulated in a given paradigm. By manipulating the cost/benefit in the same experiment, it is possible to conclude whether focusing on a reward for volunteered correct answers versus a punishment for volunteered incorrect answers produces higher metacognitive accuracy discrimination. Further, we hypothesised that punishment would lead to better metacognitive discrimination than reward. As noted above, gains/losses have been shown to impact bias; for example, a high pay-off (reward) for reporting correct information tends to produce a liberal response bias, which in and of itself can appear to increase accuracy (i.e., via higher test scores). However, because the benefit of one correct answer outweighs the cost of one incorrect answer, a high pay-off does not necessarily incentivise good metacognitive discrimination. Conversely, a high loss (punishment) should incentivise accurate metacognitive discrimination because the cost of incorrect answers has a profound impact on test scores. Thus, beyond promoting a conservative response bias, punishment also may lead people to be more accurate at distinguishing between correct and incorrect answers.

OVERVIEW OF THE EXPERIMENT Koriat and Goldsmith’s (1996; Goldsmith & Koriat, 2008) monitoring-control framework stipulates that there are three main components contributing to performance—retrieval, monitoring and control. A crucial tenet of the framework

is that the control mechanism is open to influence from a variety of factors, such as the pay-off or penalty for reporting correct and incorrect information (Higham, 2007; Koriat & Goldsmith, 1996). The current experiment was designed to explore this aspect of the control mechanism using a type-2 signal detection theory (SDT) methodology. A key benefit of type-2 SDT methodology is that it produces concrete measures of metacognitive response bias (e.g., type-2 C) and metacognitive accuracy discrimination (e.g., type-2 d′). Type-2 SDT is similar to type-1 SDT (traditionally referred to simply as SDT; Green & Swets, 1966), but where type-1 typically involves discriminating between experimenterdefined signals (e.g., studied words on a recognition test) and noise (e.g., novel words), type-2 SDT requires people to discriminate between their own correct (signal trial) and incorrect responses (noise trial).1 To explore the impact of punishment and reward on strategic regulation, we used one-pass report/withhold general knowledge tests (Arnold et al., 2013; Higham, 2007). This type of test requires participants to answer every question, but they are allowed to choose whether to volunteer (report) each answer for scoring. The benefit of the one-pass report/withhold paradigm is that it provides all necessary information to calculate the type-2 performance measures (i.e., answer accuracy for reported and withheld test trials), but it does not introduce systematic processing differences (compared to leaving questions unanswered and returning to them on a second pass; Higham, 2007). All participants completed three tests: (1) a test with a high pay-off for reporting correct answers (reward), (2) a test with a high penalty for reporting incorrect answers (punishment), and (3) a test with an equal pay-off/ penalty for correct and incorrect answers (baseline). In line with previous research, we expected participants to be significantly more conservative (i.e., withhold more answers) on the punishment than reward test; however, the novel prediction was that participants would show better 1 In-depth discussion of type-2 SDT is beyond the scope of this paper, but interested readers can refer to Higham (2007; see also Arnold et al., 2013; Galvin, Podd, Drga, & Whitmore, 2003; Higham & Arnold 2007; Higham, Perfect, & Bruno, 2009).

POSITIVE IMPACT OF PUNISHMENT

metacognitive discrimination on the punishment than reward test.

METHOD

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

Participants Eighty Flinders University undergraduates participated in Experiment 1A and 48 participated in Experiment 1B in exchange for course credit in an introductory psychology course or a $15 reimbursement. Sample size for 1A was preset to fall in the range of 76–80 participants, which was based on previous experiments with similar onepass report/withhold designs (Arnold, 2013; Arnold et al., 2013). One participant was unable to finish all three tests, and computer error produced missing data cells for two participants; thus, 77 participants contributed data to the analyses. Based on data from 1A for the key comparison of interest (i.e., d′), sample size for 1B was set to fall between 40 and 50 participants. The d′ and C measures could not be computed for one participant (perfect test score), and one paid participant was excluded for not meeting the participant restriction of being an undergraduate; thus, 46 participants contributed data to the analyses.

Design and materials The design, materials and procedure for Experiment 1A and 1B were identical, and the goal of 1B simply was to test the robustness of the findings from 1A. There were only two minor differences between the two experiments: Experiment 1B was run over a year after 1A, and different experimenters tested 1A and 1B. The experiments were a 2 (order: reward first, punishment first) × 3 (test: reward, baseline, punishment) mixed-model design, with order as the between-subjects factors. The points/penalty scoring system for the tests was as follows: +4/–1 for

the reward test, + 1/–1 for the baseline test and +1/–4 for the punishment test. The factor of four for the reward and punishment tests was used simply to orient participants to “high pay-off” (reward) versus “high penalty” (punishment). The baseline test always appeared as the third test, which means it was not a true/pure baseline condition. However, we use the term baseline simply to connote that there was no points/ penalty advantage for reported answers—one incorrect reported answer cancelled out one correct reported answer. Three general knowledge tests of similar difficulty were created, each with 35 four-alternative response questions that were originally sourced from Nelson and Narens (1980) and Arnold et al. (2013). Information regarding question difficulty was obtained from prior experiments in our laboratory that had used these questions with an undergraduate population. To counterbalance across the tests, three sets of 35 questions were constructed, with each set appearing equally often across participants in the reward, baseline and punishment tests. The tests were administrated via computer, and the format for each question consisted of four parts (see Figure 1): (1) the four-alternative question, (2) answer confidence, (3) the “Go for Points” (i.e., report) or “Withhold” decision and (4) decision confidence. The amount of points/penalties for each test was always presented in the “Go For Points/Withhold” section, which produced the only format difference between the three tests. Both answer confidence and decision confidence were rated on a 6-point scale that ranged from 0% to 100% (with increasing increments of 20). Because the focus of the current paper is on the strategic regulation, the confidence data are not presented or discussed here; however, in the interest of clarity and full disclosure of the paradigm, information about the confidence ratings are included in all relevant parts of the methods (see Arnold, 2013, for discussion of answer vs. decision confidence).

1. What city hosted the 2012 Olympic Games? Beijing

London

Sydney

Athens

Answer confidence

Choose

Figure 1. An example test question from the punishment test.

3

Go For Points (+1/–4)

Withhold (0 points)

Decision confidence

Choose

4

ARNOLD, CHISHOLM, PRIKE

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

Procedure Participants were tested in sessions of one or two people. The two-person sessions received the instructions for the first test together, but then were placed in different lab rooms to complete the experiment. The instructions for the first test explained the one-pass report/withhold format to participants; that is, an answer was required for every question, but that they were allowed to choose either to report or withhold their answer. Participants were told their goal was to achieve the highest score possible on the test and thus they should consider the amount of points/penalties for reported correct and incorrect answers, respectively, because their test score would be calculated by subtracting any penalties incurred from the points earned. The instructions then explained the two confidence ratings; specifically, that for each question they would be required to rate separately their confidence in the answer they chose and their confidence in their go for points/withhold decision. After each test was completed an on-screen prompt directed participants to alert the experimenter for further instructions. All three tests followed the same procedure, except that at the start of the second and third tests the experimenter explicitly highlighted that the next test would have a change in points/penalties, and participants were given a “reminder” sheet about the new scoring system to place beside the computer. For example, a participant who completed the reward test first would be informed prior to the start of the second (punishment) test that they now would receive +1 for a reported correct response and –4 for a reported incorrect response.

RESULTS All analyses initially were run with the order variable included; however, order had no significant impact on any of the performance measures, all Fs < 1, and thus it was not included in the reported analyses. It also was important to demonstrate that overall type-1 accuracy (i.e., ignoring report/withhold decision) did not differ between the three tests, and thus the proportion of correct answers was analysed with a withinsubjects analysis of variance (ANOVA). The results showed there was no significant difference in overall accuracy sensitivity between the reward

(1A: M = .70, SEM = .01; 1B: M = .67, SEM = .02), baseline (1A: M = .70, SEM = .01; 1B: M = .69, SEM = .02) and punishment tests (1A: M = .70, SEM = .01; 1B: M = .68, SEM = .02), for either Experiment 1A, F < 1, or 1B, F (2, 90) = 1.19, MSE = .005, p = .31, g2p = .03. Therefore, if there are differences found at the type-2 (i.e., metacognitive) level, they cannot be ascribed to differences in accuracy sensitivity at the type-1 level: The reward/penalty/baseline manipulation had no significant impact on participants’ sensitivity for choosing correct responses. The type-2 hit rate (HR) and false alarm rate (FAR) were constructed from the accuracy and report/withhold data. Specifically, in type-2 SDT, the HR for this paradigm is equal to the number of correct answers in the “go for points” column divided by the total number (i.e., reported and withheld) of correct answers, whereas the FAR is equal to the number of incorrect answers in the “go for points” column divided by the total number of incorrect answers. Type-2 mean response bias (C) and discrimination accuracy (d′) were calculated from the HR and FAR, with a correction of 1/2n added to proportions of 0 and subtracted from proportions of 1 (Macmillan & Creelman, 2005). As can be seen in Figure 2, the response bias data appear to show that, as predicted, the scoring system influenced participants’ willingness to volunteer test answers. A within-subjects ANOVA confirmed this prediction for both Experiment 1A, F (2, 152) = 28.36, MSE = .18, p < .001, g2p = .27, and 1B, F (2, 90) = 36.08, MSE = .11, p < .001, g2p = .44. There also was a significant linear trend for both Experiment 1A, F (1, 76) = 46.14, MSE = .21, p < .001, g2p = .38, and 1B, F (1, 45) = 57.68, MSE = .13, p < .001, g2p = .56, which indicated that participants’ willingness to report their answers decreases from reward to baseline to penalty. Further, a paired-samples t-test confirmed the hypothesis that response bias would differ between the reward and punishment tests: Participants were significantly more likely to report their answers on the reward test than on the penalty test in both Experiment 1A, t (76) = 6.79, p < .001, d = .83, and 1B, t (45) = 7.59, p < .001, d = 1.13. A more substantive finding than differences in response bias would be differences in discrimination accuracy. A within-subjects ANOVA confirmed that, as shown in Figure 3, monitoring ability (d′) did indeed differ across the tests for Experiment 1A, F (2, 152) = 3.53, MSE = .33,

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

POSITIVE IMPACT OF PUNISHMENT

5

Figure 2. Type-2 response bias (C) across the reward, baseline and punishment conditions for Experiment 1A (2A) and 1B (2B). Error bars depict standard error of the means.

Figure 3. Type-2 accuracy discrimination (d′) across the reward, baseline and punishment conditions for Experiment 1A (3A) and 1B (3B). Error bars depict standard error of the means.

p = .03, g2p = .04, but this pattern across the three tests was not significant for 1B, F (2, 90) = 1.76, MSE = .24, p = .18, g2p = .04. Similar to response bias, there was a significant linear trend for both Experiment 1A, F (1, 76) = 6.50, MSE = .36, p = .01, g2p = .08, and 1B, F (1, 45) = 4.16, MSE = .19, p = .05, g2p = .08, which suggests that participants’ ability to accurately monitor their own knowledge increases as the scoring system moves from reward to baseline to punishment. Finally, a paired-samples t-test supported the prediction that monitoring would be better on the punishment than the reward test: Participants were better at discriminating between their correct and incorrect answers on the punishment test than on the reward test for both Experiment 1A, t (76) = 2.55, p = .01, d = .41, and 1B, t (45) = 2.04, p = .05, d = .33. To further unpack the difference found for monitoring accuracy, the mean type-2 HR and FAR were analysed in a 2 (response type: HR, FAR) × 2 (test: reward, punishment) withinsubjects ANOVA. The baseline test was not included in the analysis simply because our

specific interest was to explore the paired-samples t-test difference in type-2 d′ reported above, however the data for all three tests are presented in Figure 4. For both experiments, there were main effects of response type, 1A: F (1, 76) = 505.01, MSE = .03, p < .001, g2p = .87; 1B: F (1, 45) = 326.20, MSE = .02, p < .001, g2p = .88, and test, 1A: F (1, 76) = 45.01, MSE = .04, p < .001, g2p = .37; 1B: F (1, 45) = 49.61, MSE = .02, p < .001, g2p = .52. However, the main effects were superseded by a significant interaction in both Experiment 1A, F (1, 76) = 8.86, MSE = .02, p = .004, g2p = .10, and 1B, F (1, 45) = 5.16, MSE = .01, p = .03, g2p = .10. The interaction for both 1A and 1B occurred because the difference in the FAR between the reward and punishment tests (1A and 1B: Mdiff = .20, SEM = .03) was larger than the difference in the HR (1A: Mdiff = .10, SEM = .02; 1B: Mdiff = .12, SEM = .02). That is, participants reported both fewer hits and false alarms on the punishment test (i.e., more conservative response bias), but this rate of withholding was significantly higher for the false alarms (i.e., higher accuracy discrimination).

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

6

ARNOLD, CHISHOLM, PRIKE

Figure 4. The type-2 HR and FAR across the reward, baseline and punishment tests for Experiment 1A (4A) and 1B (4B). Error bars depict standard error of the means.

GENERAL DISCUSSION The current study replicated the typical finding that people are more conservative in reporting their responses when there is a high penalty for volunteering incorrect information (Higham 2007; Higham & Arnold, 2007; Koriat & Goldsmith, 1994, 1996). Importantly, the study also produced the novel finding that punishing incorrect reported answers—compared to rewarding correct reported answers—led to the most accurate metacognitive discrimination. Specifically, participants were less willing to volunteer their answers on the punishment test, but this conservatism was due in part to the withholding of more incorrect information than on the reward test. To the best of our knowledge, this study is the first to directly compare reward versus punishment outcomes for the strategic regulation of accuracy. The advantage of using this type of paradigm is that it allows us to conclude that if the goal in a given situation is to promote accurate metacognitive discrimination, then

punishing people for volunteering incorrect information is more effective than rewarding them for volunteering correct information. Indeed, as demonstrated by Figures 2 and 3, any benefit of rewarding correct information, compared to no incentive (baseline) or punishment, would be produced through response bias. That is, people are more willing to provide answers when there is a large pay-off for correct information, and because the gains (points) outweigh the losses (penalty), this more liberal response bias should result in higher test scores (i.e., corrected scores). Full consideration of the current data shows, though, that it would be a mistake to necessarily equate higher test scores with accuracy precision: Rewarding correct answers led participants to volunteer more information (both correct and incorrect), but it did not prompt them to more carefully monitor the accuracy of their answers. One reason we predicted punishment may produce superior metacognitive discrimination is because the high cost for incorrect information incentivises the quality of volunteered information, whereas the high gains for correct information on the reward test incentivise the quantity of volunteered responses. Importantly, though, the better discrimination we found for the punishment test was not accompanied by a profound reduction in high-quality output. Thus, at least in the current paradigm, the effectiveness of punishment is not driven by participants setting such a conservative response bias that only a few highquality answers (e.g., those on the extreme high end of certainty/confidence) “pass” the criterion for reporting a response. However, further research is needed to compare punishment and reward across different situations/contexts, because some previous research has shown that a high penalty can produce larger drops in quantity output (e.g., Koriat & Goldsmith, 1994). Related to the above point, the positive impact of punishment on strategic regulation may be ripe for wide-spread application, especially in areas where training effective metacognitive discrimination is vital—such as medicine and business (Croskerry, 2003; Rao, 2009; Ritter, 2006). For example, medical schools use a variety of methods, such as cutoff scores (e.g., minimum 80% score to pass), to set standards that reflect the high-stakes nature of the field (Cusimano & Rothman, 2003; Richter Lagha, Boscardin, May, & Fung, 2012). Further, when educational institutions, such as law and medical schools, do assess

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

POSITIVE IMPACT OF PUNISHMENT

knowledge with a scoring system that penalises test-takers for volunteering incorrect information, the penalty typically is only a fraction of what they gain for volunteering correct information (e.g., a correction-for-guessing system; Higham, 2007; Thurstone, 1919). However, the current data highlight the potential benefit of training and testing with trial-by-trial punishment for incorrect information; for instance, instituting punishment for volunteering incorrect information from the first day of medical training to help future doctors develop better metacognitive discrimination. Additional research is warranted before any firm conclusions can be drawn about the positive impact this type of training (i.e., vs. a points > penalty system) may have on strategic regulation, but it is important to investigate any strategy that may increase efficacy and reduce costly mistakes (e.g., misdiagnoses). It will be important for future research to look across both different situations (e.g., level of knowledge/skill) and different types of materials to better understand metacognitive performance under punishment versus reward conditions. Further, although we argued that the differences found in the present study were due to whether quantity versus quality of responses was emphasised by the scoring system, further work is needed to support this interpretation. Nevertheless, in the current context the data clearly showed that punishing people for volunteering incorrect information was a more effective approach for promoting metacognitive accuracy discrimination than rewarding correct information.

REFERENCES Arnold, M. M. (2013). Monitoring and meta-metacognition in the own-race bias. Acta Psychologica, 144, 380–389. doi:10.1016/j.actpsy.2013.07.007 Arnold, M. M., Higham, P. A., & Martín-Luengo, B. (2013). A little bias goes a long way: The effects of feedback on the strategic regulation of accuracy on formula-scored tests. Journal of Experimental Psychology: Applied, 19, 383–402. doi:10.1037/a0034833 Croskerry, P. (2003). The importance of cognitive errors in diagnosis and strategies to minimize them. Academic Medicine, 78, 775–780. doi:10.1097/00001 888-200308000-00003 Cusimano, M. D., & Rothman, A. I. (2003). The effect of incorporating normative data into a criterionreferenced standard setting in medical education. Academic Medicine, 78, S88–S90. doi:10.1097/ 00001888-200310001-00028 Galvin, S. J., Podd, J. V., Drga, V., & Whitmore, J. (2003). Type 2 tasks in the theory of signal

7

detectability: Discrimination between correct and incorrect decisions. Psychonomic Bulletin & Review, 10, 843–876. doi:10.3758/BF03196546 Goldsmith, G., & Koriat, A. (2008). The strategic regulation of memory accuracy and informativeness. In A. Benjamin & B. Ross (Eds.), The psychology of learning and motivation. Vol. 48: Memory use as skilled cognition (pp. 307–324). San Diego, CA: Elsevier. Green, D., & Swets, J. (1966). Signal detection theory and psychophysics. New York, NY: Wiley. Higham, P. A. (2007). No special K! A signal-detection framework for the strategic regulation of memory accuracy. Journal of Experimental Psychology: General, 136, 1–22. doi:10.1037/0096-3445.136.1.1 Higham, P. A., & Arnold, M. M. (2007). How many questions should I answer? Using bias profiles to estimate optimal bias and maximum score on formula-scored tests. European Journal of Cognitive Psychology, 19, 718–742. doi:10.1080/0954144070 1326121 Higham, P. A., Perfect, T. J., & Bruno, D. (2009). Investigating strength and frequency effects in recognition memory using type-2 signal detection theory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 57–80. doi:10.1037/ a0013865 Koriat, A., & Goldsmith, M. (1994). Memory in naturalistic and laboratory contexts: Distinguishing the accuracy-oriented and quantity-oriented approaches to memory assessment. Journal of Experimental Psychology: General, 123, 297–315. doi:10.1037/0096-3445.123.3.297 Koriat, A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103, 490– 510. doi:10.1037//0033-295X.103.3.490 Macmillan, N. A., & Creelman, C. D. (2005). Detection theory: A user’s guide (2nd ed.). Mahwah, NJ: Lawrence Erlbaum. Nelson, T. O., & Narens, L. (1980). Norms of 300 general-information questions: Accuracy of recall, latency of recall, and feeling-of-knowing ratings. Journal of Verbal Learning and Verbal Behavior, 19, 338–368. doi:10.1016/S0022-5371 (80)90266-2 Rao, G. (2009). Probability error in diagnosis: The conjunction fallacy among beginning medical students. Medical Student Education, 41, 262–265. Richter Lagha, R. A., Boscardin, C. K., May, W., & Fung, C. (2012). A comparison of two standardsetting approaches in high-stakes clinical performance assessment using generalizability theory. Academic Medicine, 87, 1077–1082. doi:10.1097/ACM. 0b013e31825cea4b Ritter, B. A. (2006). Can business ethics be trained? A study of the ethical decision-making process in business students. Journal of Business Ethics, 68, 153–164. doi:10.1007/s10551-006-9062-0 Roebers, C. M., & Fernandez, O. (2002). The effects of accuracy motivation on children’s and adults’ event recall, suggestibility, and their answers to unanswerable questions. Journal of Cognition and Development, 3, 415–443. doi:10.1080/15248372.2002.9669676

8

ARNOLD, CHISHOLM, PRIKE

Downloaded by [University of Lethbridge] at 11:27 11 November 2015

Roebers, C. M., & Schneider, W. (2005). The strategic regulation of children’s memory performance and suggestibility. Journal of Experimental Child Psychology, 91, 24–44. doi:10.1016/j.jecp.2005.01.001 Roebers, C. M., Moga, N., & Schneider, W. (2001). The role of accuracy motivation on children’s and adults’

event recall. Journal of Experimental Child Psychology, 78, 313–329. doi:10.1006/jecp.2000.2577 Thurstone, L. L. (1919). A scoring method for mental tests. Psychological Bulletin, 16, 235–240. doi:10. 1037/h0069898

No pain no gain: The positive impact of punishment on the strategic regulation of accuracy.

Previous studies have shown that punishing people through a large penalty for volunteering incorrect information typically leads them to withhold more...
397KB Sizes 0 Downloads 6 Views