Memory, 2015 Vol. 23, No. 2, 119–126, http://dx.doi.org/10.1080/09658211.2013.870210
Age-related differences in episodic feeling-of-knowing arise from differences in memory performance Mathilde Sacher, Lionel Landré, and Laurence Taconnat UMR CNRS 7295, Centre de Recherche sur la Cognition et l’Apprentissage, Université François Rabelais, Tours, France
(Received 21 January 2013; accepted 25 November 2013)
Previous studies about the effects of ageing on the episodic feeling-of-knowing (FOK) accuracy and its underlying processes have yielded conflicting results. Recent work suggests that using alternative measures to gamma correlations might allow more accurate and informative interpretations of metamemory performance in ageing. We therefore investigated this issue with a large sample of 59 young and 61 older participants using alternative signal-detection theory (SDT) measures. These measures (receiver operating characteristic curves and Brier score) are recommended in the literature and able to reveal the characteristic profile of impairment in ageing. Our results suggest that the FOK accuracy deficit observed in the literature arises from differences in memory performance. This observation provides a convenient explanation to the previous discrepancies and furthermore supports the interest of the use of SDT-derived measures in the frame of metamemory.
Keywords: Metamemory; Feeling-of-knowing; Ageing; Episodic memory; Signal-detection theory.
Metamemory refers to people’s knowledge, monitoring and control of their memory processes (Nelson & Narens, 1990). Feeling-of-knowing (FOK) corresponds to the subjective experience whereby one fails to recall information, yet it is confident that the information is still available in memory and will be retrieved in the future. This metamemory judgement, based on the monitoring of the memory retrieval process, plays a central role in regulating memory functioning. FOK judgements are experimentally assessed by a procedure in which participants are asked to estimate the likelihood that they will subsequently recognise a piece of information that they had failed to recall earlier, from either semantic (Hart, 1965) or episodic memory (Schacter, 1983). FOK judgements may be based on two sources of information:
information-based and experience-based judgements (Koriat, 2007; Koriat & Levy-Sadot, 1999). Information-based judgements rely on people’s beliefs about their skills, the task and the cognitive strategies (i.e., domain-specific knowledge retrieved from long-term memory). In contrast, experience-based judgements are formed on subjective feelings that are mediated by the implicit application of nonanalytic heuristics relying on a variety of mnemonics cues. These cues include the accessibility of pertinent partial information about a solicited target (Koriat, 1993), the ease with which information comes to mind (Kelley & Lindsay, 1993) and the familiarity of the cue that serves to probe memory (Reder & Schunn, 1996). In the ageing literature, no age-related differences have been found for FOK accuracy for
Address correspondence to: Mathilde Sacher, UMR CNRS 7295, Université François-Rabelais, 3 rue des Tanneurs - B.P. 4103, 37 041 Tours Cedex 1, France. E-mail: [email protected] Current address: Lionel Landré, Laboratoire EMC (EA 3082), Université Lyon Lumière, Bron, France The authors wish to thank C. Hertzog for his very insightful comments before the submission of this manuscript.
© 2013 Taylor & Francis
120
SACHER, LANDRÉ, TACONNAT
information stored in semantic memory (e.g., Allen-Burge & Storandt, 2000). However, some studies have demonstrated a deficit in episodic FOK accuracy in older adults (Perrotin, Isingrini, Souchay, Clarys, & Taconnat, 2006; Souchay & Isingrini, 2012; Souchay, Isingrini, & Espagnet, 2000; Souchay, Moulin, Clarys, Taconnat, & Isingrini, 2007) while some others have not (Hertzog, Dunlosky, & Sinclair, 2010; MacLaverty & Hertzog, 2009). These differences in the effect of ageing on episodic FOK accuracy could be attributed to varying experimental methods, pertaining to sample characteristics (the older group covered a wide age range and a low educational level in the first group of studies), the type of material (associated cue–target pairs in the first group or unrelated cue–target pairs in the second group of studies), the type of FOK prediction assessment (binary or n-point likelihood prediction scale, respectively, in the first and second group of studies) or the type of subsequent recognition task (yes/no target recognition test or forced-choice associative test, respectively, in the first and second group of studies). Alternatively, some authors have argued that the difference in metamemory could arise from differences in memory, demonstrating that the control of memory level between the two age groups cancelled the differences in FOK accuracy (Hertzog et al., 2010). Accordingly, Souchay and Isingrini (2012) found out that controlling for memory performance using covariance analysis in a study using the design of the first group cancelled out the effects of ageing on FOK accuracy, questioning the metamnestic nature of age-related FOK accuracy decline. However, Thomas, Bulevich, and Dubois (2011) showed that an effect of ageing on FOK accuracy could still be found when the age groups were matched for memory performance, using experimental procedures similar to those of Hertzog et al. (2010). In all of these studies, FOK accuracy was assessed by intra-individual Goodman–Kruskal Gamma (G) correlations (Nelson, 1984), providing a unique quantitative index of the strength of the relationship between FOK predictions and subsequent recognition performance. However, recent studies (Benjamin & Diaz, 2008; Masson & Rotello, 2009) have indicated potential measurement issues with gamma correlations, suggesting the alternative use of a signal-detection framework for measuring metamemory accuracy. Using numeric simulations, Masson and Rotello (2009) demonstrated that the empirically determined
value of G systematically deviates from its actual value under realistic conditions. They advised that metamemory accuracy should be evaluated using measures derived from signal-detection theory (SDT), which may provide more efficient measures of biases in accuracy. They further suggested that receiver operating characteristic (ROC) curves, which assess the relative variance in signal and noise distribution, should be examined by estimating the area under the ROC curve (Az), corresponding to a measure of the overall discrimination accuracy. Furthermore, unitary measures of FOK such as gamma correlations are likely to account for a limited part of cognitive processes or the result of several indistinguishable processes at work. Schraw (2009) has therefore recently recommended using multiple measures (or multiple outcomes measures) to explore metamemory, in order to provide useful information about different facets of metacognitive monitoring. Accordingly, the discrepancies found in the literature indicate that FOK accuracy alteration could depend on specific conditions, pertaining to sample characteristics as well as experimental design or control of memory level. This observation that multiple parameters may influence this effect suggests that the effects of ageing on FOK might be best described as a profile rather than by a single, binary assessment of the whole effect. Complementary measures such as calibration curves which reflect the ways in which predictions vary with memory performance, bias which indicates participants’ over- or under-confidence, ROC-derived measures of sensitivity and specificity and the Brier score which is a composite measure of calibration accuracy identifying the respective contribution of several potential subprocesses (Keren, 1991; Lichtenstein & Fischhoff, 1977) are commonly used by researchers in related areas (e.g., education, decisionmaking). For example, the Brier score allows one to evaluate the absolute accuracy of individual FOK predictions (calibration score), the discrimination of the outcomes (resolution score) and the uncertainty of the outcome measure (knowledge score). It therefore allows to distinguish episodic FOK differences arising from the calibration process, the discrimination process and the difficulty to draw predictions due to recognition scores close to the chance level. These multiple measures may thus provide significant insights to the underlying mechanisms of episodic FOK change in ageing and shed a new light on the differences found in the literature. As
AGEING AND EPISODIC FOK ACCURACY
a consequence, the purpose of this study was to investigate the effects of ageing on episodic FOK accuracy using multiple measures recommended in the literature in a large sample of participants, in order to determine the specific pattern of episodic FOK change in ageing.
METHOD Participants From a total of 131 participants, 61 older adults (17 men, 44 women; age range 61–82, M = 69.44 years, SD = 5.72) and 59 non-student young adults (19 men, 40 women; age range 20–36, M = 26.76 years, SD = 3.96) were included for analyses in this study (see FOK measures, below, for exclusions). All the participants reported that they were in good physical and mental health, free from psychoactive medication and that they had completed eight or more years of education. Older participants were screened for possible dementia using the Mini-Mental State Examination (MMSE, Folstein, Folstein, & McHugh, 1975) with a cut-off score of 28. Ethical approval was obtained from the psychology department of the University of Tours, and all the participants signed consent forms. The older adults had significantly fewer years of education (M = 13.43, SD = 2.74) than the younger ones [M = 14.32, SD = 1.82; t(118) = 2.11, p < .05] but performed better on the Mill Hill Vocabulary Test (Deltour, 1993) [older participants: M = 29.05, SD = 3.28; younger participants: M = 27.1, SD = 2.77; t(118) = –3.51, p < .001], while the younger participants reported lower levels of subjective memory complaints [younger participants: M = 2.07, SD = .53; older participants: M = 2.41, SD = .44; t(118) = –3.79, p < .001; Questionnaire d’Auto-évaluation de la Mémoire (QAM) i.e., self-assessment memory questionnaire; Van der Linden, Wyns, Coyette, von Frenckell, & Seron, 1989].
Procedure The episodic FOK procedure and the material used here were similar to those used by Sacher, Taconnat, Souchay, and Isingrini (2009). Briefly, the participants were first presented with 60 weakly associated cue–target word pairs (e.g., customsCONTRABAND; 6 s per pair) and instructed to retain them in memory for a subsequent
121
cued-recall test. After a 30-s interference task (processing speed test), participants were shown each cue (10 s per item) and asked to recall the corresponding target word. The list was then presented again, and participants had to predict on a six-point scale (0, 20, 40, 60, 80 or 100% sure) the likelihood that they would subsequently recognise the corresponding target word among four semantically associated distractors. Immediately after this FOK phase, participants were given the recognition test (10 s per item). A five-alternative, forced-choice recognition task was administered for each of the 60 word pairs. Five alternative response words were presented, the target item plus four semantically similar distractors (e.g., FRAUD, TRAFFIC, EXCHANGE, CONTRABAND and COUNTERFEITING). Participants were asked to say the word which had previously been presented.
Memory measures Episodic memory scores were calculated as the proportion of correctly recalled target words (cued-recall score) and the proportion of correctly recognised targets (recognition score).
FOK measures FOK measures are only calculated on unrecalled items (omission errors). Five young and six older participants with a cued-recall score above 45/60 were therefore excluded from the study due to insufficient unrecalled material to compute FOK measures. Descriptive FOK measures. The magnitude of FOK represents the mean of FOK judgements per participant. Calibration curves (see Lichtenstein & Fischhoff, 1977) were plotted as the level of recognition performance achieved per FOK judgement category, with a diagonal straight line indicating perfect calibration. Calibration curves illustrate the extent to which participants tend to under- or over-estimate their recognition ability, based on their FOK judgements. FOK accuracy measures. The Goodman–Kruskal gamma correlation (G; Nelson, 1984) was computed for each participant. The gamma score provides a measure of the strength of association between two ordered variables ranging from –1
122
SACHER, LANDRÉ, TACONNAT
to 1, with positive values corresponding to a strong association between recognition performance and FOK judgements, values close to zero corresponding to no association and negative values to an inverse relationship. Due to potential measurement bias in this measure, FOK accuracy was also evaluated using measures derived from SDT. ROC curves were computed as the maximum likelihood estimates of “A” and “B” parameters of the conventional binormal model using ROCKIT 1.1B2 software. ROC curves represent the cumulative proportion of correct recognitions and incorrect recognitions per prediction category (see Benjamin & Diaz, 2008; Masson & Rotello, 2009). The area under the curve (Az) was then computed for each participant. Moreover, two parameters associated to the accuracy of ROC curves were computed. The sensitivity indicates the likelihood to have predicted the recognition of a recognised item and the specificity indicates the likelihood to have predicted the non-recognition of a non-recognised item. The Brier score (Keren, 1991; Lichtenstein & Fischhoff, 1977) was computed for each participant. This measure is related to the calibration curve and refers to the correspondence between FOK judgement and recognition performance. It can be subdivided into three parts: calibration, resolution and knowledge (Brier score = knowledge + calibration – resolution; the closer to zero, the better). Calibration is an absolute measure of prediction accuracy which represents the itembased ability to assign an appropriate FOK prediction to the corresponding recognition performance. It broadly corresponds to the distance between the calibration curve and the perfect calibration (45° line) at each point (i.e., the mean weighted difference between the FOK prediction category and the corresponding recognition rates, the lower the score the better). The Brier resolution subscore (as opposed to gamma resolution) is a discrimination index which relates to the variance of recognition rates across prediction categories (i.e., a higher and better resolution rate is achieved when recognition rates differ a lot across predictive categories, indicating a good discrimination between items that will be recognised and items that will not be recognised). However, it differs from calibration score because it does not indicate that the items were affected the correct prediction category. Conversely, knowledge is a global measure of uncertainty of the outcome representing the inherent variance of the predicted items recognition [as given by C (1–C), with C
being the mean recognition rate for all predicted items]. In the episodic FOK procedure, it reaches its minimal (and better) value when the recognition rate approaches 0 or 1, and its maximal value when the recognition rate is 0.5. It is worth noting that this score is outside the judge’s control, and depends on memory rather than metamemory processes although it influences the ease of the predictions.
RESULTS The memory (cued recall and recognition) and FOK (magnitude of FOK judgements and FOK accuracy) measures are summarised in Table 1.
Episodic memory Age had a significant impact on correct recall [F(1,118) = 41.47, MSE = .02, p < .001, g2p = .26] and correct recognition [F(1,118) = 35.03, MSE = .01, p < .001, g2p = .23], with older adults recalling and recognising fewer target words than young adults.
Description of FOK judgements The magnitude of FOK judgements was significantly lower in the older group [F(1,118) = 11.72, MSE = .04, p < .001, g2p = .09], with similar standard deviations in the two groups (Young: 0.21; Old: 0.21), indicating that older participants used lower categories of FOK judgements than younger participants. The calibration curves (see Figure 1A) indicate an underestimation in both groups for all possible FOK categories (except 100%, which cannot lead to an underestimation). However, young participants tended to display a linear relationship between all FOK ratings and the corresponding recognition scores, whereas the recognition scores of the older participants appeared not to follow their FOK predictions for low FOK ratings (0–40%) which was the most largely used prediction category for this age group (73.2%). When considering all items (both recalled and unrecalled) the difference between mean FOK predictions and mean recognition scores tended to confirm a greater underestimation of older participants’ own performance than younger participants [M = –.25, SD = .16 and M = –.16, SD = .10, respectively, F(1,118) = 13.08, MSE = .02, p < .001,
AGEING AND EPISODIC FOK ACCURACY
123
TABLE 1 Means and standard deviations for episodic memory and FOK measures by age group Younger (n = 59)
Memory measures Cued recall Correct Recognition Overall correct Omitted items correct Descriptive FOK measures Magnitude of FOK judgements FOK accuracy measures Gamma index Area under the ROC curve (Az) Accuracy Sensitivity Specificity Brier score Calibration Resolution Knowledge †
Older (n = 61)
M
SD
M
SD
F(1,118)
.50
.13
.33
.15
41.47***
.89 .79
.08 .10
.79 .70
.10 .10
35.03*** 26.04***
.48
.21
.35
.21
11.72***
.52 .67 .55 .51 .67 .30 .17 .02 .15
.30 .11 .21 .33 .35 .14 .13 .02 .05
.16 .56 .47 .31 .77 .39 .21 .02 .20
.30 .12 .15 .30 .30 .14 .14 .02 .04
41.42*** 46.93*** 7.57** 12.26*** 3.23† 12.22*** 2.59 1.09 28.7***
p < .10, **p < .01, ***p < .001.
g2p = .10]. However, when biases were calculated only on the unrecalled items, no significant difference was found between the two age groups [older participants: M = –.35, SD = .19; younger participants: M = –.31, SD = .2; F(1,118) = 1.47, NS].
Accuracy of FOK judgements Gamma index was reliably different from zero in all groups (p < .001) and was significantly lower for
Figure 1.
older than younger participants [F(1,118) = 41.42, MSE = .09, p < .001, g2p = .26]. These results indicate that the young adults evaluated their ability to recognise newly learned items more accurately than the older participants. The graphic representation of the ROC curves of the two groups reveals a potential difference in FOK accuracy, with a symmetrical and higher ROC curve for young participants, and an asymmetrical and lower curve for older participants
(A) Calibration curves and percentage of FOK judgements per category by age group. (B) ROC curves by age group.
124
SACHER, LANDRÉ, TACONNAT
(see Figure 1B). Older participants were less accurate than younger participants [F(1,118) = 7.57, MSE = .03, p < .01, g2p = .06] with less sensitive [F(1,118) = 12.26, MSE = .09, p < .001, g2p = .09], but marginally more specific predictions [F(1,118) = 3.23, MSE = .10, p = .07, g2p = .03]. A t-test confirmed a highly significant difference between the areas under the curves (Az) of the two groups [F(1,118) = 46.93, MSE = .01, p < .001, g2p = .28], indicating a lower FOK accuracy in the older group. The total Brier score was also significantly lower for young participants [F(1,118) =12.22, MSE = .02, p < .001, g2p = .09], indicating again a better correspondence between FOK judgements and recognition performance in this group. Analysis of Brier subscores revealed a significant age effect only for the knowledge index [F(1,118) = 28.7, MSE = .01, p < .001, g2p = .20], with a greater uncertainty of the recognition among older than younger participants. There was no significant difference between groups in the distance between predictions and perfect calibration (calibration index) or in the discrimination between recognised and non-recognised items (resolution index). These results indicate that differences in FOK accuracy between older and younger adults are related to the uncertainty of their recognition (lower knowledge due to a recognition rate closer to the chance level), rather than to their discrimination and absolute accuracy (equivalent resolution and calibration).
DISCUSSION The main goal of this study was to investigate the specific pattern of episodic FOK accuracy alteration in ageing, in order to enhance our understanding of age-related metacognitive changes and to clarify the discrepancies existing in the literature. This study is the first to our knowledge to use SDT-derived alternative measures to gamma correlations, providing more reliable and detailed analyses of FOK accuracy than gamma correlations alone. As expected, our ageing participants displayed a lower level of memory (as assessed by cued recall and recognition) than younger participants. Accordingly, the magnitude of FOK judgements was found lower in the older than in the younger group. Older adults also presented a stronger bias towards an under-confidence in their global recognition ability, which corroborates the observation of a higher level of memory complaint in this
group as measured by the QAM. However, when bias analyses were restricted to the unrecalled items (those which are used to calculate FOK accuracy indexes), the difference was found nonsignificant between the two age groups. Nevertheless, our ROC-derived measures of sensitivity and specificity indicated a lower sensitivity, and marginally higher specificity in older participants, which tends to support the idea of an underestimation of their recognition performance. One explanation of the discrepancy between the two measures would be that the older adults tend to use to a greater extent the lower categories, and especially the 0 and 20% chance category which can only lead to a reduction of sensitivity. Given that the recognition test is an alternative forcedchoice with five items, participants are at least expected to recognise 20% of the items by chance, thus the 0% category should not be used by any of the participants (see MacLaverty & Hertzog, 2009 for an experimental manipulation of this parameter). It seems that older people are not generally able to make this deduction on their own, as the 0% chance category was used in 23% of their judgements (55% when considering the 0 and 20% categories together, as opposed to 12 and 33%, respectively, for young participants). Thus, older adults would tend to underestimate their performance for the lower categories to a greater extent for than younger adults, due to their misuse of the scale (as confirmed by their higher calibration curve for the 0 and 20% categories, see Figure 1A). Alternatively, younger adults underestimated more their performance for higher prediction categories (as given by the higher level of their calibration curve for the 40–100% categories), which would have rendered the difference between biases non-significant. This observation both suggests that the older adults present a stronger underestimation bias than younger participants for the lowest prediction categories, possibly due to their misuse of the scale, and supports the interest of the use of sensitivity and specificity index, as well as calibration curves in interpreting the descriptive results of FOK accuracy studies. Concerning FOK accuracy, our results for gamma index as well as ROC curves and the Brier score, support the idea of a deficit in older adults, indicating that older people estimate their ability to recognise newly learned items less accurately than younger adults. These results are in line with previous observations of an age-related deficit in episodic FOK accuracy (Perrotin et al., 2006; Souchay et al., 2000; Souchay et al., 2007; Thomas
AGEING AND EPISODIC FOK ACCURACY
et al., 2011) but tend to contradict the findings of other studies in which no such effect was observed (Hertzog et al., 2010; MacLaverty & Hertzog, 2009; Souchay & Isingrini, 2012). The fact that all of our measures of FOK accuracy demonstrate an impairment in the older group does not allow us to confirm or infirm Masson & Rotello’s (2009) arguments for the use of other measures than gamma correlations. However, our results for the Brier score, showing that knowledge, but not calibration or resolution, is impaired in ageing suggest that the ageing effect on episodic FOK accuracy arises from global differences in the uncertainty of the outcome, rather than an item-specific discrimination or prediction deficit (as demonstrated by the absence of impaired resolution and calibration). The knowledge index depends directly on the total recognition rate of predicted items (the closer to 0.5, the higher) and is only affected by memory processes and external factors such as the difficulty of the recognition task, which are out of the assessor’s control (Keren, 1991). In turn, the resolution subscore depends on the ability of the assessor to distinguish between subsequently recognised and non-recognised items, and calibration measures the fitness between the predictions and the outcome. This result indicates that the ageing effect we found on FOK accuracy would derive directly from differences in memory performance between age groups. Parameters such as the relative task difficulty, as given by the recognition score, may therefore explain some of the age group differences in FOK accuracy. The effects of ageing on metamemory would then partly arise from the effects of age-related memory alteration, as mediated by the reduced predictability of items recognition. This hypothesis would provide a convenient explanation to the fact that some studies controlling for memory performance either a priori, by manipulating experimental conditions across groups (Hertzog et al., 2010) or a posteriori, by regressing out the effect of memory (Souchay & Isingrini, 2012) hardly find an ageing effect on FOK accuracy. On the other hand, two recent studies have separately demonstrated age-related episodic FOK accuracy differences even though memory was equated between groups (Thomas et al., 2011), and metamemory invariance even though memory performance differed between young and older participants (Eakin, Hertzog, & Harris, in press). Taken together with the previous studies and
125
ours, these results suggest that metamemory performance in older participants depends on both memory-related and metamemory-specific processes. The effect of those memory-dependent and memory-independent processes can be separately assessed as suggested by Maniscalco and Lau (2012) for metamemory measures such as confidence judgements accuracy, by estimating the optimal SDT characteristics during the first step (i.e., recognition) and partialling it out of the second step (confidence judgements). However, this procedure does not directly fit prospective judgements such as the FOK paradigm, in which the recognition phase comes after the judgement. The Brier subscores provide an equivalent and adequate way to determine the relative part of memory-dependent (knowledge) and memoryindependent (resolution and calibration) processes in FOK accuracy assessment. Furthermore, the advantage of using the Brier score to partial out the effect of memory over the method of Hertzog et al. (2010) would be that it does not require using different experimental procedures between groups, which could have influenced metamemory as well as memory. As compared to the covariance procedure used in the study of Souchay and Isingrini (2012), the use of Brier score would allow to partial out directly the individual effect of memory, instead of regressing the estimated inter-individual effect of memory with potential covariates such as age or executive functioning. In sum, the present study suggests that FOK accuracy deficit in older adults mainly derives from an increased uncertainty of the outcome, as given by their memory performance. This observation provides a fair explanation of the discrepancies observed in the literature, and further suggests that SDT-derived measures such as the Brier score should be used in future studies to partial out the effect of memory on metamemory accuracy.
References Allen-Burge, R., & Storandt, M. (2000). Age equivalence in feeling-of-knowing experiences. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 55, 214–223. doi:10.1093/geronb/ 55.4.P214 Benjamin, A. S., & Diaz, M. (2008). Measurement of relative metamnemonic accuracy. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of memory and metamemory (pp. 195–210). New York, NY: Psychology Press.
126
SACHER, LANDRÉ, TACONNAT
Deltour, J. J. (1993). Echelle de vocabulaire de Mill Hill de J. C. Raven. Adaptation française et normes européennes du Mill Hill et du Standard Progressive Matrices de Raven (PM 38) [Mill Hill vocabulary scale of J.C. Raven. French adaptation and European norms for Mill Hill and Raven’s Progressive Matrices (PM 38)]. Brainele-Château: Editions L’application des techniques modernes. Eakin, D. K., Hertzog, C., & Harris, W. (in press). Age invariance in semantic and episodic metamemory: Both younger and older adults provide accurate feeling-of-knowing for names of faces. Aging, Neuropsychology, and Cognition: A Journal on Normal and Dysfunctional Development, 21(1), 27–51. doi:10.1080/13825585.2013.775217 Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189–198. doi:10.1016/0022-3956(75)90026-6 Hart, J. T. (1965). Memory and the feeling-of-knowing experience. Journal of Educational Psychology, 56, 208–216. doi:10.1037/h0022263 Hertzog, C., Dunlosky, J., & Sinclair, S. M. (2010). Episodic feeling-of-knowing resolution derives from the quality of original encoding. Memory & Cognition, 38, 771–784.doi:10.3758/MC.38.6.771 Kelley, C. M., & Lindsay, D. S. (1993). Remembering mistaken for knowing: Ease of retrieval as a basis for confidence in answers to general knowledge questions. Journal of Memory and Language, 32(1), 1–24. doi:10.1006/jmla.1993.1001 Keren, G. (1991). Calibration and probability judgements: Conceptual and methodological issues. Acta Psychologica, 77, 217–273. doi:10.1016/0001-6918(91) 90036-Y Koriat, A. (1993). How do we know that we know? The accessibility model of the feeling-of-knowing. Psychological Review, 100, 609–639. doi:10.1037/0033295X.100.4.609 Koriat, A. (2007). Metacognition and consciousness. In P. D. Zelazo, M. Moscovitch, & E. Thompson (Eds.), Cambridge handbook of consciousness (pp. 289–325). New York, NY: Cambridge University Press. Koriat, A., & Levy-Sadot, R. (1999). Processes underlying metacognitive judgments: Information-based and experience-based monitoring of one’s own knowledge. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology (pp. 483–502). New York, NY: Guilford Press. Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how, much they know? Organizational Behavior and Human Performance, 20, 159–183. doi:10.1016/0030-5073(77) 90001-0 MacLaverty, S. N., & Hertzog, C. (2009). Do agerelated differences in episodic feeling of knowing accuracy depend on the timing of the judgement? Memory, 17, 860–873. doi:10.1080/096582109033 74537 Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive
sensitivity from confidence ratings. Consciousness and Cognition, 21, 422–430. doi:10.1016/j. concog.2011.09.021 Masson, M. E. J., & Rotello, C. M. (2009). Sources of bias in the Goodman–Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 509–527. doi:10.1037/a0014876 Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95(1), 109–133. doi:10.1037/ 0033-2909.95.1.109 Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (pp. 125–173). New York, NY: Academic Press. Perrotin, A., Isingrini, M., Souchay, C., Clarys, D., & Taconnat, L. (2006). Episodic feeling of knowing accuracy and cued recall in the elderly: Evidence for double dissociation involving executive functioning and processing speed. Acta Psychologica, 122(1), 58–73. doi:10.1016/j.actpsy.2005.10.003 Reder, L. M., & Schunn, C. D. (1996). Metacognition does not imply awareness: Strategy choice is governed by implicit learning and memory. In L. M. Reder (Ed.), Implicit memory and metacognition (pp. 45–78). Hillsdale, NJ: Erlbaum. Sacher, M., Taconnat, L., Souchay, C., & Isingrini, M. (2009). Divided attention at encoding: Effect on feeling-of-knowing. Consciousness and Cognition, 18, 754–761. doi:10.1016/j.concog.2009.04.001 Schacter, D. L. (1983). Feeling-of-knowing in episodic memory. Journal of Experimental Psychology: Learning, Memory and Cognition, 9(1), 39–54. doi:10.1037/ 0278-7393.9.1.39 Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning, 4(1), 33–45. doi:10.1007/s11409008-9031-3 Souchay, C., & Isingrini, M. (2012). Are feeling-ofknowing and judgment-of-learning different? Evidence from older adults. Acta Psychologica, 139, 458–464. doi:10.1016/j.actpsy.2012.01.007 Souchay, C., Isingrini, M., & Espagnet, L. (2000). Aging, episodic memory feeling-of-knowing, and frontal functioning. Neuropsychology, 14, 299–309. doi:10.1037/0894-4105.14.2.299 Souchay, C., Moulin, C. J. A., Clarys, D., Taconnat, L., & Isingrini, M. (2007). Diminished episodic memory awareness in older adults: Evidence from feeling-ofknowing and recollection. Consciousness and Cognition, 16, 769–784. doi:10.1016/j.concog.2006.11.002 Thomas, A. K., Bulevich, J. B., & Dubois, S. J. (2011). Context affects feeling-of-knowing accuracy in younger and older adults. Journal of Experimental Psychology: Learning, Memory and Cognition, 37(1), 96–108. doi:10.1037/a0021612 Van der Linden, M., Wyns, C., Coyette, F., von Frenckell, R., & Seron, X. (1989). Questionnaire d’Auto-évaluation de la Mémoire [Self-assessment memory questionnaire]. Bruxelles: Editest.
Copyright of Memory is the property of Psychology Press (UK) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
Age-related differences in episodic feeling-of-knowing arise from differences in memory performance Mathilde Sacher, Lionel Landré, and Laurence Taconnat UMR CNRS 7295, Centre de Recherche sur la Cognition et l’Apprentissage, Université François Rabelais, Tours, France
(Received 21 January 2013; accepted 25 November 2013)
Previous studies about the effects of ageing on the episodic feeling-of-knowing (FOK) accuracy and its underlying processes have yielded conflicting results. Recent work suggests that using alternative measures to gamma correlations might allow more accurate and informative interpretations of metamemory performance in ageing. We therefore investigated this issue with a large sample of 59 young and 61 older participants using alternative signal-detection theory (SDT) measures. These measures (receiver operating characteristic curves and Brier score) are recommended in the literature and able to reveal the characteristic profile of impairment in ageing. Our results suggest that the FOK accuracy deficit observed in the literature arises from differences in memory performance. This observation provides a convenient explanation to the previous discrepancies and furthermore supports the interest of the use of SDT-derived measures in the frame of metamemory.
Keywords: Metamemory; Feeling-of-knowing; Ageing; Episodic memory; Signal-detection theory.
Metamemory refers to people’s knowledge, monitoring and control of their memory processes (Nelson & Narens, 1990). Feeling-of-knowing (FOK) corresponds to the subjective experience whereby one fails to recall information, yet it is confident that the information is still available in memory and will be retrieved in the future. This metamemory judgement, based on the monitoring of the memory retrieval process, plays a central role in regulating memory functioning. FOK judgements are experimentally assessed by a procedure in which participants are asked to estimate the likelihood that they will subsequently recognise a piece of information that they had failed to recall earlier, from either semantic (Hart, 1965) or episodic memory (Schacter, 1983). FOK judgements may be based on two sources of information:
information-based and experience-based judgements (Koriat, 2007; Koriat & Levy-Sadot, 1999). Information-based judgements rely on people’s beliefs about their skills, the task and the cognitive strategies (i.e., domain-specific knowledge retrieved from long-term memory). In contrast, experience-based judgements are formed on subjective feelings that are mediated by the implicit application of nonanalytic heuristics relying on a variety of mnemonics cues. These cues include the accessibility of pertinent partial information about a solicited target (Koriat, 1993), the ease with which information comes to mind (Kelley & Lindsay, 1993) and the familiarity of the cue that serves to probe memory (Reder & Schunn, 1996). In the ageing literature, no age-related differences have been found for FOK accuracy for
Address correspondence to: Mathilde Sacher, UMR CNRS 7295, Université François-Rabelais, 3 rue des Tanneurs - B.P. 4103, 37 041 Tours Cedex 1, France. E-mail: [email protected] Current address: Lionel Landré, Laboratoire EMC (EA 3082), Université Lyon Lumière, Bron, France The authors wish to thank C. Hertzog for his very insightful comments before the submission of this manuscript.
© 2013 Taylor & Francis
120
SACHER, LANDRÉ, TACONNAT
information stored in semantic memory (e.g., Allen-Burge & Storandt, 2000). However, some studies have demonstrated a deficit in episodic FOK accuracy in older adults (Perrotin, Isingrini, Souchay, Clarys, & Taconnat, 2006; Souchay & Isingrini, 2012; Souchay, Isingrini, & Espagnet, 2000; Souchay, Moulin, Clarys, Taconnat, & Isingrini, 2007) while some others have not (Hertzog, Dunlosky, & Sinclair, 2010; MacLaverty & Hertzog, 2009). These differences in the effect of ageing on episodic FOK accuracy could be attributed to varying experimental methods, pertaining to sample characteristics (the older group covered a wide age range and a low educational level in the first group of studies), the type of material (associated cue–target pairs in the first group or unrelated cue–target pairs in the second group of studies), the type of FOK prediction assessment (binary or n-point likelihood prediction scale, respectively, in the first and second group of studies) or the type of subsequent recognition task (yes/no target recognition test or forced-choice associative test, respectively, in the first and second group of studies). Alternatively, some authors have argued that the difference in metamemory could arise from differences in memory, demonstrating that the control of memory level between the two age groups cancelled the differences in FOK accuracy (Hertzog et al., 2010). Accordingly, Souchay and Isingrini (2012) found out that controlling for memory performance using covariance analysis in a study using the design of the first group cancelled out the effects of ageing on FOK accuracy, questioning the metamnestic nature of age-related FOK accuracy decline. However, Thomas, Bulevich, and Dubois (2011) showed that an effect of ageing on FOK accuracy could still be found when the age groups were matched for memory performance, using experimental procedures similar to those of Hertzog et al. (2010). In all of these studies, FOK accuracy was assessed by intra-individual Goodman–Kruskal Gamma (G) correlations (Nelson, 1984), providing a unique quantitative index of the strength of the relationship between FOK predictions and subsequent recognition performance. However, recent studies (Benjamin & Diaz, 2008; Masson & Rotello, 2009) have indicated potential measurement issues with gamma correlations, suggesting the alternative use of a signal-detection framework for measuring metamemory accuracy. Using numeric simulations, Masson and Rotello (2009) demonstrated that the empirically determined
value of G systematically deviates from its actual value under realistic conditions. They advised that metamemory accuracy should be evaluated using measures derived from signal-detection theory (SDT), which may provide more efficient measures of biases in accuracy. They further suggested that receiver operating characteristic (ROC) curves, which assess the relative variance in signal and noise distribution, should be examined by estimating the area under the ROC curve (Az), corresponding to a measure of the overall discrimination accuracy. Furthermore, unitary measures of FOK such as gamma correlations are likely to account for a limited part of cognitive processes or the result of several indistinguishable processes at work. Schraw (2009) has therefore recently recommended using multiple measures (or multiple outcomes measures) to explore metamemory, in order to provide useful information about different facets of metacognitive monitoring. Accordingly, the discrepancies found in the literature indicate that FOK accuracy alteration could depend on specific conditions, pertaining to sample characteristics as well as experimental design or control of memory level. This observation that multiple parameters may influence this effect suggests that the effects of ageing on FOK might be best described as a profile rather than by a single, binary assessment of the whole effect. Complementary measures such as calibration curves which reflect the ways in which predictions vary with memory performance, bias which indicates participants’ over- or under-confidence, ROC-derived measures of sensitivity and specificity and the Brier score which is a composite measure of calibration accuracy identifying the respective contribution of several potential subprocesses (Keren, 1991; Lichtenstein & Fischhoff, 1977) are commonly used by researchers in related areas (e.g., education, decisionmaking). For example, the Brier score allows one to evaluate the absolute accuracy of individual FOK predictions (calibration score), the discrimination of the outcomes (resolution score) and the uncertainty of the outcome measure (knowledge score). It therefore allows to distinguish episodic FOK differences arising from the calibration process, the discrimination process and the difficulty to draw predictions due to recognition scores close to the chance level. These multiple measures may thus provide significant insights to the underlying mechanisms of episodic FOK change in ageing and shed a new light on the differences found in the literature. As
AGEING AND EPISODIC FOK ACCURACY
a consequence, the purpose of this study was to investigate the effects of ageing on episodic FOK accuracy using multiple measures recommended in the literature in a large sample of participants, in order to determine the specific pattern of episodic FOK change in ageing.
METHOD Participants From a total of 131 participants, 61 older adults (17 men, 44 women; age range 61–82, M = 69.44 years, SD = 5.72) and 59 non-student young adults (19 men, 40 women; age range 20–36, M = 26.76 years, SD = 3.96) were included for analyses in this study (see FOK measures, below, for exclusions). All the participants reported that they were in good physical and mental health, free from psychoactive medication and that they had completed eight or more years of education. Older participants were screened for possible dementia using the Mini-Mental State Examination (MMSE, Folstein, Folstein, & McHugh, 1975) with a cut-off score of 28. Ethical approval was obtained from the psychology department of the University of Tours, and all the participants signed consent forms. The older adults had significantly fewer years of education (M = 13.43, SD = 2.74) than the younger ones [M = 14.32, SD = 1.82; t(118) = 2.11, p < .05] but performed better on the Mill Hill Vocabulary Test (Deltour, 1993) [older participants: M = 29.05, SD = 3.28; younger participants: M = 27.1, SD = 2.77; t(118) = –3.51, p < .001], while the younger participants reported lower levels of subjective memory complaints [younger participants: M = 2.07, SD = .53; older participants: M = 2.41, SD = .44; t(118) = –3.79, p < .001; Questionnaire d’Auto-évaluation de la Mémoire (QAM) i.e., self-assessment memory questionnaire; Van der Linden, Wyns, Coyette, von Frenckell, & Seron, 1989].
Procedure The episodic FOK procedure and the material used here were similar to those used by Sacher, Taconnat, Souchay, and Isingrini (2009). Briefly, the participants were first presented with 60 weakly associated cue–target word pairs (e.g., customsCONTRABAND; 6 s per pair) and instructed to retain them in memory for a subsequent
121
cued-recall test. After a 30-s interference task (processing speed test), participants were shown each cue (10 s per item) and asked to recall the corresponding target word. The list was then presented again, and participants had to predict on a six-point scale (0, 20, 40, 60, 80 or 100% sure) the likelihood that they would subsequently recognise the corresponding target word among four semantically associated distractors. Immediately after this FOK phase, participants were given the recognition test (10 s per item). A five-alternative, forced-choice recognition task was administered for each of the 60 word pairs. Five alternative response words were presented, the target item plus four semantically similar distractors (e.g., FRAUD, TRAFFIC, EXCHANGE, CONTRABAND and COUNTERFEITING). Participants were asked to say the word which had previously been presented.
Memory measures Episodic memory scores were calculated as the proportion of correctly recalled target words (cued-recall score) and the proportion of correctly recognised targets (recognition score).
FOK measures FOK measures are only calculated on unrecalled items (omission errors). Five young and six older participants with a cued-recall score above 45/60 were therefore excluded from the study due to insufficient unrecalled material to compute FOK measures. Descriptive FOK measures. The magnitude of FOK represents the mean of FOK judgements per participant. Calibration curves (see Lichtenstein & Fischhoff, 1977) were plotted as the level of recognition performance achieved per FOK judgement category, with a diagonal straight line indicating perfect calibration. Calibration curves illustrate the extent to which participants tend to under- or over-estimate their recognition ability, based on their FOK judgements. FOK accuracy measures. The Goodman–Kruskal gamma correlation (G; Nelson, 1984) was computed for each participant. The gamma score provides a measure of the strength of association between two ordered variables ranging from –1
122
SACHER, LANDRÉ, TACONNAT
to 1, with positive values corresponding to a strong association between recognition performance and FOK judgements, values close to zero corresponding to no association and negative values to an inverse relationship. Due to potential measurement bias in this measure, FOK accuracy was also evaluated using measures derived from SDT. ROC curves were computed as the maximum likelihood estimates of “A” and “B” parameters of the conventional binormal model using ROCKIT 1.1B2 software. ROC curves represent the cumulative proportion of correct recognitions and incorrect recognitions per prediction category (see Benjamin & Diaz, 2008; Masson & Rotello, 2009). The area under the curve (Az) was then computed for each participant. Moreover, two parameters associated to the accuracy of ROC curves were computed. The sensitivity indicates the likelihood to have predicted the recognition of a recognised item and the specificity indicates the likelihood to have predicted the non-recognition of a non-recognised item. The Brier score (Keren, 1991; Lichtenstein & Fischhoff, 1977) was computed for each participant. This measure is related to the calibration curve and refers to the correspondence between FOK judgement and recognition performance. It can be subdivided into three parts: calibration, resolution and knowledge (Brier score = knowledge + calibration – resolution; the closer to zero, the better). Calibration is an absolute measure of prediction accuracy which represents the itembased ability to assign an appropriate FOK prediction to the corresponding recognition performance. It broadly corresponds to the distance between the calibration curve and the perfect calibration (45° line) at each point (i.e., the mean weighted difference between the FOK prediction category and the corresponding recognition rates, the lower the score the better). The Brier resolution subscore (as opposed to gamma resolution) is a discrimination index which relates to the variance of recognition rates across prediction categories (i.e., a higher and better resolution rate is achieved when recognition rates differ a lot across predictive categories, indicating a good discrimination between items that will be recognised and items that will not be recognised). However, it differs from calibration score because it does not indicate that the items were affected the correct prediction category. Conversely, knowledge is a global measure of uncertainty of the outcome representing the inherent variance of the predicted items recognition [as given by C (1–C), with C
being the mean recognition rate for all predicted items]. In the episodic FOK procedure, it reaches its minimal (and better) value when the recognition rate approaches 0 or 1, and its maximal value when the recognition rate is 0.5. It is worth noting that this score is outside the judge’s control, and depends on memory rather than metamemory processes although it influences the ease of the predictions.
RESULTS The memory (cued recall and recognition) and FOK (magnitude of FOK judgements and FOK accuracy) measures are summarised in Table 1.
Episodic memory Age had a significant impact on correct recall [F(1,118) = 41.47, MSE = .02, p < .001, g2p = .26] and correct recognition [F(1,118) = 35.03, MSE = .01, p < .001, g2p = .23], with older adults recalling and recognising fewer target words than young adults.
Description of FOK judgements The magnitude of FOK judgements was significantly lower in the older group [F(1,118) = 11.72, MSE = .04, p < .001, g2p = .09], with similar standard deviations in the two groups (Young: 0.21; Old: 0.21), indicating that older participants used lower categories of FOK judgements than younger participants. The calibration curves (see Figure 1A) indicate an underestimation in both groups for all possible FOK categories (except 100%, which cannot lead to an underestimation). However, young participants tended to display a linear relationship between all FOK ratings and the corresponding recognition scores, whereas the recognition scores of the older participants appeared not to follow their FOK predictions for low FOK ratings (0–40%) which was the most largely used prediction category for this age group (73.2%). When considering all items (both recalled and unrecalled) the difference between mean FOK predictions and mean recognition scores tended to confirm a greater underestimation of older participants’ own performance than younger participants [M = –.25, SD = .16 and M = –.16, SD = .10, respectively, F(1,118) = 13.08, MSE = .02, p < .001,
AGEING AND EPISODIC FOK ACCURACY
123
TABLE 1 Means and standard deviations for episodic memory and FOK measures by age group Younger (n = 59)
Memory measures Cued recall Correct Recognition Overall correct Omitted items correct Descriptive FOK measures Magnitude of FOK judgements FOK accuracy measures Gamma index Area under the ROC curve (Az) Accuracy Sensitivity Specificity Brier score Calibration Resolution Knowledge †
Older (n = 61)
M
SD
M
SD
F(1,118)
.50
.13
.33
.15
41.47***
.89 .79
.08 .10
.79 .70
.10 .10
35.03*** 26.04***
.48
.21
.35
.21
11.72***
.52 .67 .55 .51 .67 .30 .17 .02 .15
.30 .11 .21 .33 .35 .14 .13 .02 .05
.16 .56 .47 .31 .77 .39 .21 .02 .20
.30 .12 .15 .30 .30 .14 .14 .02 .04
41.42*** 46.93*** 7.57** 12.26*** 3.23† 12.22*** 2.59 1.09 28.7***
p < .10, **p < .01, ***p < .001.
g2p = .10]. However, when biases were calculated only on the unrecalled items, no significant difference was found between the two age groups [older participants: M = –.35, SD = .19; younger participants: M = –.31, SD = .2; F(1,118) = 1.47, NS].
Accuracy of FOK judgements Gamma index was reliably different from zero in all groups (p < .001) and was significantly lower for
Figure 1.
older than younger participants [F(1,118) = 41.42, MSE = .09, p < .001, g2p = .26]. These results indicate that the young adults evaluated their ability to recognise newly learned items more accurately than the older participants. The graphic representation of the ROC curves of the two groups reveals a potential difference in FOK accuracy, with a symmetrical and higher ROC curve for young participants, and an asymmetrical and lower curve for older participants
(A) Calibration curves and percentage of FOK judgements per category by age group. (B) ROC curves by age group.
124
SACHER, LANDRÉ, TACONNAT
(see Figure 1B). Older participants were less accurate than younger participants [F(1,118) = 7.57, MSE = .03, p < .01, g2p = .06] with less sensitive [F(1,118) = 12.26, MSE = .09, p < .001, g2p = .09], but marginally more specific predictions [F(1,118) = 3.23, MSE = .10, p = .07, g2p = .03]. A t-test confirmed a highly significant difference between the areas under the curves (Az) of the two groups [F(1,118) = 46.93, MSE = .01, p < .001, g2p = .28], indicating a lower FOK accuracy in the older group. The total Brier score was also significantly lower for young participants [F(1,118) =12.22, MSE = .02, p < .001, g2p = .09], indicating again a better correspondence between FOK judgements and recognition performance in this group. Analysis of Brier subscores revealed a significant age effect only for the knowledge index [F(1,118) = 28.7, MSE = .01, p < .001, g2p = .20], with a greater uncertainty of the recognition among older than younger participants. There was no significant difference between groups in the distance between predictions and perfect calibration (calibration index) or in the discrimination between recognised and non-recognised items (resolution index). These results indicate that differences in FOK accuracy between older and younger adults are related to the uncertainty of their recognition (lower knowledge due to a recognition rate closer to the chance level), rather than to their discrimination and absolute accuracy (equivalent resolution and calibration).
DISCUSSION The main goal of this study was to investigate the specific pattern of episodic FOK accuracy alteration in ageing, in order to enhance our understanding of age-related metacognitive changes and to clarify the discrepancies existing in the literature. This study is the first to our knowledge to use SDT-derived alternative measures to gamma correlations, providing more reliable and detailed analyses of FOK accuracy than gamma correlations alone. As expected, our ageing participants displayed a lower level of memory (as assessed by cued recall and recognition) than younger participants. Accordingly, the magnitude of FOK judgements was found lower in the older than in the younger group. Older adults also presented a stronger bias towards an under-confidence in their global recognition ability, which corroborates the observation of a higher level of memory complaint in this
group as measured by the QAM. However, when bias analyses were restricted to the unrecalled items (those which are used to calculate FOK accuracy indexes), the difference was found nonsignificant between the two age groups. Nevertheless, our ROC-derived measures of sensitivity and specificity indicated a lower sensitivity, and marginally higher specificity in older participants, which tends to support the idea of an underestimation of their recognition performance. One explanation of the discrepancy between the two measures would be that the older adults tend to use to a greater extent the lower categories, and especially the 0 and 20% chance category which can only lead to a reduction of sensitivity. Given that the recognition test is an alternative forcedchoice with five items, participants are at least expected to recognise 20% of the items by chance, thus the 0% category should not be used by any of the participants (see MacLaverty & Hertzog, 2009 for an experimental manipulation of this parameter). It seems that older people are not generally able to make this deduction on their own, as the 0% chance category was used in 23% of their judgements (55% when considering the 0 and 20% categories together, as opposed to 12 and 33%, respectively, for young participants). Thus, older adults would tend to underestimate their performance for the lower categories to a greater extent for than younger adults, due to their misuse of the scale (as confirmed by their higher calibration curve for the 0 and 20% categories, see Figure 1A). Alternatively, younger adults underestimated more their performance for higher prediction categories (as given by the higher level of their calibration curve for the 40–100% categories), which would have rendered the difference between biases non-significant. This observation both suggests that the older adults present a stronger underestimation bias than younger participants for the lowest prediction categories, possibly due to their misuse of the scale, and supports the interest of the use of sensitivity and specificity index, as well as calibration curves in interpreting the descriptive results of FOK accuracy studies. Concerning FOK accuracy, our results for gamma index as well as ROC curves and the Brier score, support the idea of a deficit in older adults, indicating that older people estimate their ability to recognise newly learned items less accurately than younger adults. These results are in line with previous observations of an age-related deficit in episodic FOK accuracy (Perrotin et al., 2006; Souchay et al., 2000; Souchay et al., 2007; Thomas
AGEING AND EPISODIC FOK ACCURACY
et al., 2011) but tend to contradict the findings of other studies in which no such effect was observed (Hertzog et al., 2010; MacLaverty & Hertzog, 2009; Souchay & Isingrini, 2012). The fact that all of our measures of FOK accuracy demonstrate an impairment in the older group does not allow us to confirm or infirm Masson & Rotello’s (2009) arguments for the use of other measures than gamma correlations. However, our results for the Brier score, showing that knowledge, but not calibration or resolution, is impaired in ageing suggest that the ageing effect on episodic FOK accuracy arises from global differences in the uncertainty of the outcome, rather than an item-specific discrimination or prediction deficit (as demonstrated by the absence of impaired resolution and calibration). The knowledge index depends directly on the total recognition rate of predicted items (the closer to 0.5, the higher) and is only affected by memory processes and external factors such as the difficulty of the recognition task, which are out of the assessor’s control (Keren, 1991). In turn, the resolution subscore depends on the ability of the assessor to distinguish between subsequently recognised and non-recognised items, and calibration measures the fitness between the predictions and the outcome. This result indicates that the ageing effect we found on FOK accuracy would derive directly from differences in memory performance between age groups. Parameters such as the relative task difficulty, as given by the recognition score, may therefore explain some of the age group differences in FOK accuracy. The effects of ageing on metamemory would then partly arise from the effects of age-related memory alteration, as mediated by the reduced predictability of items recognition. This hypothesis would provide a convenient explanation to the fact that some studies controlling for memory performance either a priori, by manipulating experimental conditions across groups (Hertzog et al., 2010) or a posteriori, by regressing out the effect of memory (Souchay & Isingrini, 2012) hardly find an ageing effect on FOK accuracy. On the other hand, two recent studies have separately demonstrated age-related episodic FOK accuracy differences even though memory was equated between groups (Thomas et al., 2011), and metamemory invariance even though memory performance differed between young and older participants (Eakin, Hertzog, & Harris, in press). Taken together with the previous studies and
125
ours, these results suggest that metamemory performance in older participants depends on both memory-related and metamemory-specific processes. The effect of those memory-dependent and memory-independent processes can be separately assessed as suggested by Maniscalco and Lau (2012) for metamemory measures such as confidence judgements accuracy, by estimating the optimal SDT characteristics during the first step (i.e., recognition) and partialling it out of the second step (confidence judgements). However, this procedure does not directly fit prospective judgements such as the FOK paradigm, in which the recognition phase comes after the judgement. The Brier subscores provide an equivalent and adequate way to determine the relative part of memory-dependent (knowledge) and memoryindependent (resolution and calibration) processes in FOK accuracy assessment. Furthermore, the advantage of using the Brier score to partial out the effect of memory over the method of Hertzog et al. (2010) would be that it does not require using different experimental procedures between groups, which could have influenced metamemory as well as memory. As compared to the covariance procedure used in the study of Souchay and Isingrini (2012), the use of Brier score would allow to partial out directly the individual effect of memory, instead of regressing the estimated inter-individual effect of memory with potential covariates such as age or executive functioning. In sum, the present study suggests that FOK accuracy deficit in older adults mainly derives from an increased uncertainty of the outcome, as given by their memory performance. This observation provides a fair explanation of the discrepancies observed in the literature, and further suggests that SDT-derived measures such as the Brier score should be used in future studies to partial out the effect of memory on metamemory accuracy.
References Allen-Burge, R., & Storandt, M. (2000). Age equivalence in feeling-of-knowing experiences. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 55, 214–223. doi:10.1093/geronb/ 55.4.P214 Benjamin, A. S., & Diaz, M. (2008). Measurement of relative metamnemonic accuracy. In J. Dunlosky & R. A. Bjork (Eds.), Handbook of memory and metamemory (pp. 195–210). New York, NY: Psychology Press.
126
SACHER, LANDRÉ, TACONNAT
Deltour, J. J. (1993). Echelle de vocabulaire de Mill Hill de J. C. Raven. Adaptation française et normes européennes du Mill Hill et du Standard Progressive Matrices de Raven (PM 38) [Mill Hill vocabulary scale of J.C. Raven. French adaptation and European norms for Mill Hill and Raven’s Progressive Matrices (PM 38)]. Brainele-Château: Editions L’application des techniques modernes. Eakin, D. K., Hertzog, C., & Harris, W. (in press). Age invariance in semantic and episodic metamemory: Both younger and older adults provide accurate feeling-of-knowing for names of faces. Aging, Neuropsychology, and Cognition: A Journal on Normal and Dysfunctional Development, 21(1), 27–51. doi:10.1080/13825585.2013.775217 Folstein, M. F., Folstein, S. E., & McHugh, P. R. (1975). Mini-mental state: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189–198. doi:10.1016/0022-3956(75)90026-6 Hart, J. T. (1965). Memory and the feeling-of-knowing experience. Journal of Educational Psychology, 56, 208–216. doi:10.1037/h0022263 Hertzog, C., Dunlosky, J., & Sinclair, S. M. (2010). Episodic feeling-of-knowing resolution derives from the quality of original encoding. Memory & Cognition, 38, 771–784.doi:10.3758/MC.38.6.771 Kelley, C. M., & Lindsay, D. S. (1993). Remembering mistaken for knowing: Ease of retrieval as a basis for confidence in answers to general knowledge questions. Journal of Memory and Language, 32(1), 1–24. doi:10.1006/jmla.1993.1001 Keren, G. (1991). Calibration and probability judgements: Conceptual and methodological issues. Acta Psychologica, 77, 217–273. doi:10.1016/0001-6918(91) 90036-Y Koriat, A. (1993). How do we know that we know? The accessibility model of the feeling-of-knowing. Psychological Review, 100, 609–639. doi:10.1037/0033295X.100.4.609 Koriat, A. (2007). Metacognition and consciousness. In P. D. Zelazo, M. Moscovitch, & E. Thompson (Eds.), Cambridge handbook of consciousness (pp. 289–325). New York, NY: Cambridge University Press. Koriat, A., & Levy-Sadot, R. (1999). Processes underlying metacognitive judgments: Information-based and experience-based monitoring of one’s own knowledge. In S. Chaiken & Y. Trope (Eds.), Dual process theories in social psychology (pp. 483–502). New York, NY: Guilford Press. Lichtenstein, S., & Fischhoff, B. (1977). Do those who know more also know more about how, much they know? Organizational Behavior and Human Performance, 20, 159–183. doi:10.1016/0030-5073(77) 90001-0 MacLaverty, S. N., & Hertzog, C. (2009). Do agerelated differences in episodic feeling of knowing accuracy depend on the timing of the judgement? Memory, 17, 860–873. doi:10.1080/096582109033 74537 Maniscalco, B., & Lau, H. (2012). A signal detection theoretic approach for estimating metacognitive
sensitivity from confidence ratings. Consciousness and Cognition, 21, 422–430. doi:10.1016/j. concog.2011.09.021 Masson, M. E. J., & Rotello, C. M. (2009). Sources of bias in the Goodman–Kruskal gamma coefficient measure of association: Implications for studies of metacognitive processes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 35, 509–527. doi:10.1037/a0014876 Nelson, T. O. (1984). A comparison of current measures of the accuracy of feeling-of-knowing predictions. Psychological Bulletin, 95(1), 109–133. doi:10.1037/ 0033-2909.95.1.109 Nelson, T. O., & Narens, L. (1990). Metamemory: A theoretical framework and new findings. In G. H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (pp. 125–173). New York, NY: Academic Press. Perrotin, A., Isingrini, M., Souchay, C., Clarys, D., & Taconnat, L. (2006). Episodic feeling of knowing accuracy and cued recall in the elderly: Evidence for double dissociation involving executive functioning and processing speed. Acta Psychologica, 122(1), 58–73. doi:10.1016/j.actpsy.2005.10.003 Reder, L. M., & Schunn, C. D. (1996). Metacognition does not imply awareness: Strategy choice is governed by implicit learning and memory. In L. M. Reder (Ed.), Implicit memory and metacognition (pp. 45–78). Hillsdale, NJ: Erlbaum. Sacher, M., Taconnat, L., Souchay, C., & Isingrini, M. (2009). Divided attention at encoding: Effect on feeling-of-knowing. Consciousness and Cognition, 18, 754–761. doi:10.1016/j.concog.2009.04.001 Schacter, D. L. (1983). Feeling-of-knowing in episodic memory. Journal of Experimental Psychology: Learning, Memory and Cognition, 9(1), 39–54. doi:10.1037/ 0278-7393.9.1.39 Schraw, G. (2009). A conceptual analysis of five measures of metacognitive monitoring. Metacognition and Learning, 4(1), 33–45. doi:10.1007/s11409008-9031-3 Souchay, C., & Isingrini, M. (2012). Are feeling-ofknowing and judgment-of-learning different? Evidence from older adults. Acta Psychologica, 139, 458–464. doi:10.1016/j.actpsy.2012.01.007 Souchay, C., Isingrini, M., & Espagnet, L. (2000). Aging, episodic memory feeling-of-knowing, and frontal functioning. Neuropsychology, 14, 299–309. doi:10.1037/0894-4105.14.2.299 Souchay, C., Moulin, C. J. A., Clarys, D., Taconnat, L., & Isingrini, M. (2007). Diminished episodic memory awareness in older adults: Evidence from feeling-ofknowing and recollection. Consciousness and Cognition, 16, 769–784. doi:10.1016/j.concog.2006.11.002 Thomas, A. K., Bulevich, J. B., & Dubois, S. J. (2011). Context affects feeling-of-knowing accuracy in younger and older adults. Journal of Experimental Psychology: Learning, Memory and Cognition, 37(1), 96–108. doi:10.1037/a0021612 Van der Linden, M., Wyns, C., Coyette, F., von Frenckell, R., & Seron, X. (1989). Questionnaire d’Auto-évaluation de la Mémoire [Self-assessment memory questionnaire]. Bruxelles: Editest.
Copyright of Memory is the property of Psychology Press (UK) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.
