Behav Res DOI 10.3758/s13428-014-0492-9

Russian normative data for 375 action pictures and verbs Yulia Akinina & Svetlana Malyutina & Maria Ivanova & Ekaterina Iskra & Elena Mannova & Olga Dragoy

# Psychonomic Society, Inc. 2014

Abstract The present article introduces a Russian-language database of 375 action pictures and associated verbs with normative data. The pictures were normed for name agreement, conceptual familiarity, and subjective visual complexity, and measures of age of acquisition, imageability, and image agreement were collected for the verbs. Values of objective visual complexity, as well as information about verb frequency, length, argument structure, instrumentality, and name relation, are also provided. Correlations between these parameters are presented, along with a comparative analysis of the Russian name agreement norms and those collected in other languages. The full set of pictorial stimuli and the obtained norms may be freely downloaded from http://neuroling.ru/en/ db.htm for use in research and for clinical purposes. Keywords Russian norms . Action pictures . Verbs Since the appearance of the seminal normative studies of Lachman (1973) and Snodgrass and Vanderwart (1980), numerous normative databases of noun–object stimuli have been created with different psycholinguistic parameters. For nouns, databases have been developed for a variety of languages (for English: Barry, Morrison, & Ellis, 1997; Berman, Friedman, Y. Akinina (*) : M. Ivanova : E. Iskra : O. Dragoy National Research University Higher School of Economics, Moscow, Russia e-mail: [email protected] S. Malyutina University of South Carolina, Columbia, SC, USA E. Iskra : E. Mannova Center for Speech Pathology and Neurorehabilitation, Moscow, Russia O. Dragoy Moscow Research Institute of Psychiatry, Moscow, Russia

Hamberger, & Snodgrass, 1989; Himmanen, Gentles, & Sailor, 2003; Salmon, McMullen, & Filliter, 2010; Snodgrass & Vanderwart, 1980; for French: Alario & Ferrand, 1999; Bonin, Peereman, Malardier, Méot, & Chalard, 2003; for European Spanish: Cuetos, Ellis, & Alvarez, 1999; Sanfeliu & Fernandez, 1996; for Argentinian Spanish: Manoiloff, Arstein, Bélen Canavoso, Fernández, & Segui, 2010; for Italian: Dell’Acqua, Lotto, & Job, 2000; Nisi, Longoni, & Snodgrass, 2000; for German: Schröder, Gemballa, Ruppin, & Wartenburger, 2012; for Modern Greek: Dimitropoulou, Duñabeitia, Blitsas, & Carreiras, 2009; for Dutch: Martein, 1995; for Icelandic: Pind, Jónsdóttir, Gissurardóttir, & Jónsson, 2000; for Russian: Tsaparina, Bonin, & Méot, 2011; for Japanese: Nishimoto, Miyawaki, Ueda, Une, & Takahashi, 2005; Nishimoto, Ueda, Miyawaki, Une, & Takahashi, 2012; for Chinese: Weekes, Shu, Hao, Liu, & Tan, 2007; for English, German, Spanish, Italian, Bulgarian, Hungarian, and Mandarin Chinese: Bates et al., 2003), and for different age groups (for children: Berman, Friedman, Hamberger, & Snodgrass, 1989; Cycowicz, Friedman, Rothstein, & Snodgrass, 1997; for elderly people: Cuetos, Samartino, & Ellis, 2012). Databases using the same visual stimuli, often taken (wholly or partially) from the set of black-andwhite line drawings published by Snodgrass and Vanderwart (1980; e.g., Alario & Ferrand, 1999; Bates et al., 2003; Manoiloff et al., 2010; Nisi et al., 2000; Pind et al., 2000; Sanfeliu & Fernandez, 1996; Weekes et al., 2007) or its colorized version (Rossion & Pourtois, 2004; e.g., Dimitropoulou et al., 2009; Tsaparina et al., 2011; Weekes et al., 2007), allow for direct cross-linguistic comparisons. Cross-linguistic studies are also carried out on the basis of databases including pictures that were originally developed or selected from different sources (Alario & Ferrand, 1999; Bates et al., 2003; Kremin et al., 2003; Székely et al., 2004; Nishimoto et al., 2005). The databases

Behav Res

of standardized norms provide values for different parameters (such as name and image agreement, picture visual complexity, conceptual familiarity, word age of acquisition, imageability, etc.) that can be used to balance materials in psycholinguistic experiments, can serve as independent variables in various experimental designs, or may be of interest by themselves, since they demonstrate the statistical interrelations among each other, shedding light on the mental lexicon’s structure and its functioning. Also, knowledge about the relevant psycholinguistic parameters of items is crucial when language assessment and therapeutic tools are being developed for clinical purposes. Following extensive normative data collection for object pictures and associated nouns in different languages and populations, an interest has recently emerged in the standardization of action pictures and verbs. This has largely been due to the growing recognition of the critical role of verbs in language disorders. Verb production is particularly vulnerable following brain damage that results in the breakdown of language, a condition known as aphasia (Bastiaanse, 1991). A common symptom of so-called agrammatic aphasia (Goodglass, 1993), verb deficits have also been reported for other aphasia types (Jonkers & Bastiaanse, 2007; Luzzatti, Aggujaro, & Crepaldi, 2006; Mätzig, Druks, Masterson, & Vigliocco, 2009, for a review). Careful matching for different psycholinguistic properties, such as word frequency, age of acquisition, and picture complexity, has not been able to completely eliminate a processing disadvantage for action naming as compared to object naming, in both healthy and brain-lesioned populations (Mätzig et al., 2009; Szekely et al., 2005), which supports the idea that specific brain resources or greater resource recruitment are allocated to verb processing. Lesions studies correlating dysfunction of particular brain regions with specific linguistic deficits have identified the frontal and, to a lesser degree, the parietal lobes of the left hemisphere as the main loci of verb production (see Cappa & Perani, 2003, for a review). However, growing evidence is suggesting that this view is overly simplistic. Some individuals with verb impairments have lesions outside of the left frontal regions, and some with vast frontal lesions have verb production that is relatively spared (see Crepaldi, Berlingeri, Paulesu, & Luzzatti, 2011, for a review). Also, some neurodegenerative diseases—for example, different types of frontotemporal dementia—also show impaired action naming relative to object naming (Cotelli et al., 2006). All the same, some patient groups may not show this pattern (e.g., with semantic dementia, see Cotelli et al., 2006). Furthermore, the verb–noun discrepancy effect may be eliminated in some patients when certain aspects of stimuli are controlled. For example, d’Honincthun and Pillon (2008) found that verb naming and comprehension deficits were no longer found in one patient with a frontal variant of frontotemporal dementia when naming and comprehension were assessed using

videotaped actions or verbal stimuli, rather than depicted actions (i.e., photographs). The pervasiveness of verb deficits in patients with different neurological syndromes and brain lesion topography necessitates the establishment of a reliable research apparatus, as well as accurate clinical tools for verb assessment, which can take into consideration normative information about verbs and associated pictorial stimuli. A number of parameters have been identified that may contribute to lexical processing, and to verb production in particular (e.g., in an action-naming task), and must therefore be controlled for in aphasia assessment batteries and carefully manipulated in experimental aphasia studies. The parameters characterize a concrete stimulus—a word or a corresponding picture. Some of these parameters are obtained in experiments in which participants are asked to perform a task involving the stimulus—for instance, to name a picture in order to obtain name agreement scores. Some of the parameters are subjective and are obtained in questionnaires in which participants are asked to rate the stimulus on a particular scale. For instance, image agreement, imageability, visual complexity, familiarity, and age of acquisition are rated parameters, and the numbers of points on scales vary among studies. Some parameters can be taken from specialized databases (e.g., frequency), and some can be directly observed—for example, word length. Finally, specific word-class parameters are determined by experts in linguistics (for verbs, such parameters include argument structure, instrumentality, and name relation). All of the parameters mentioned above will be defined and discussed below. Name agreement is a measure of the uniformity of names used by different participants to refer to a depicted object or action. Two measures for name agreement are commonly used. The percentage of name agreement (%NA) is the number of participants (per hundred) who gave the most frequent response in a picture-naming task. Another measure, the H statistic, is computed by the following formula: k

H ¼

i¼1

X

pi log2

1 ; pi

where k is the number of different names given to each picture, and pi is the proportion of respondents who gave each name (Snodgrass & Vanderwart, 1980). The H statistic has been widely used in naming studies since Snodgrass and Vanderwart’s, because this measure provides information about the distribution of responses across participants. Whereas %NA reflects the homogeneity of naming responses, the H statistic pertains to the heterogeneity of the naming behavior of participants. Name agreement has been associated with robust effects in naming. Vitkovitch and Tyrrell (1995) found that objects with higher name agreement are recognized in an object decision task as quickly as those with multiple correct names, but more quickly than objects that are often assigned

Behav Res

erroneous names. In a naming task, objects with high name agreement were named more quickly than either of two sets with low name agreement scores. This suggests that name agreement affects the lexical access that takes place after structural recognition. Naming latencies in object and action naming also increase with decreased name agreement scores (Alario et al., 2004; Barry et al., 1997; Bonin, Boyer, Méot, Fayol, & Droit, 2004; Bonin et al., 2003; Cuetos & Alija, 2003; Ellis & Morrison, 1998; Nishimoto et al., 2012; Weekes et al., 2007). In addition, in individuals with aphasia, name agreement is a strong determinant of action-naming accuracy (Kemmerer & Tranel, 2000). Image agreement is the degree to which a visually depicted object or action corresponds to the mental image generated in response to the presentation of a word. High image agreement scores tend to contribute to shorter object- and action-naming times in healthy populations (Barry et al., 1997; Bonin et al., 2004; Bonin et al., 2003; Nishimoto et al., 2012) and to better action-naming performance in individuals with brain damage (Kemmerer & Tranel, 2000). It is generally assumed that higher image agreement scores pertain to the presence of canonical mental images (Barry et al., 1997). Conversely, at the level of object recognition, lower image agreement (i.e., an image being less similar to one’s mental representation) leads to slower recognition (e.g., Alario et al., 2004; Barry et al., 1997). Imageability refers to the degree of effort with which a mental image of a corresponding object or action can be generated in response to a verbal stimulus. It is thought to indicate the semantic richness of a concept. Higher imageability, and therefore higher semantic richness, may facilitate phonological access in primed word-naming task (Cortese, Simpson, & Woolsey, 1997). The imageability effect, which manifests in reduced naming times for moreimageable words, has been replicated in different studies using noun stimuli (Alario et al., 2004; Ellis & Morrison, 1998; Nickels & Howard, 1995; but see Nishimoto et al., 2012). However, for verbs, the effect is inconsistent, and hence the role of imageability is unclear. Shao, Roelofs, and Meyer (2013) found that imageability was a significant predictor of action-naming latencies; Cuetos and Alija (2003) reported a similar effect that approached significance; and Bonin, Boyer, Méot, Fayol, and Droit (2004) found no reliable effect. Several studies have made a direct comparison of the imageability parameters in verbs and nouns. Chiarello, Shears, and Lund (1999) reported stronger correspondence between imageability ratings and reaction times for nouns than for verbs, which could imply easier mental image generation for nouns or a lesser importance of this parameter for verb processing. In a study by Masterson and Druks (1998), verbs obtained significantly lower imageability ratings than did nouns. At the same time, some studies (e.g., Bird, Howard, & Franklin, 2003) have showen that when

imageability ratings were controlled for, the verb–noun dissociation in the imageability scores disappeared. This allowed researchers to draw the conclusion that, at least in some cases, verb–noun discrepancies could be reduced to imageability effects, although this view was criticized (for a review, see Druks, 2002). Connell and Lynott (2012) showed that while rating imageability, the participants tended to rely on the visual modality (for nouns), which, in the authors’ opinion, was prompted by the formulation of the task “generate a mental image.” Modalities other than the visual modality might contribute more to the mental representations of verbs (e.g., they may be associated with motor actions). The described inconsistent effects and considerations regarding imageability warrant further investigation of its role in verb processing and the inclusion of this parameter in verbnorming studies. Overall, this variable should still be controlled in clinical research. Visual complexity may refer either to subjective ratings of the amount of detail in a picture or to objective characteristics of the digitized image—for instance, the size of the file in different formats (Szekely & Bates 2000). Visual complexity is thought to affect recognition times, and therefore may then affect naming speed. Several studies have shown subjective visual complexity to be a predictor of object-naming latencies in healthy individuals (Alario et al., 2004; Ellis & Morrison, 1998), and accuracy in people with aphasia (e.g., Cuetos, Aguado, Izura, & Ellis, 2002), although in many studies no reliable effect of this parameter on object-naming latencies has been found (e.g., Barry et al., 1997; Bonin et al., 2003; Cuetos, Ellis, & Alvarez, 1999; Snodgrass & Yuditsky, 1996; Weekes et al., 2007). One of the problems with this measure is related to the fact that subjective visual complexity ratings rely on behavioral performance, and may be confounded by subjective familiarity, word frequency, and age of acquisition. To resolve this issue, Székely and Bates analyzed several objective measures of digitized visual stimuli, and found that the PDF, TIFF, and JPG formats provided valid values of objective visual complexity that strongly correlated with subjective ratings and—unlike the subjective ratings—affected naming accuracy, although with no effect on naming times in their study. Familiarity is a conceptual parameter pertaining to the estimated degree to which the depicted object or action is familiar to participants—that is, how often they think about or deal with the object or the action. During the conceptual familiarity task, participants can be asked to make their ratings on the basis of the presented word (e.g., Cuetos & Alija, 2003; Ellis & Morrison, 1998), as well as of the picture (e.g., Bonin et al., 2004; Fiez & Tranel, 1997; Snodgrass & Vanderwart 1980); in the latter case, the instruction is to estimate the concept, and not the depicted version of it. A familiarity effect has been reported in some studies: the concepts for more familiar objects are processed faster (Cuetos et al., 1999;

Behav Res

Snodgrass & Yuditsky, 1996), although in other studies a small or an absent effect has been found (see Alario et al., 2004, for a comparative analysis of eight naming studies in six languages). Familiarity has been shown to affect objectnaming performance in people with aphasia (Cuetos et al., 2002). In Kemmerer and Tranel (2000), familiarity was shown to be a reliable determinant of verb production accuracy in an action-naming task in a large group of participants with brain damage. Taking into account the discrepancies in the measurements and the inconsistency of the results, the role of familiarity in lexical retrieval during picture naming remains unclear (Schröder et al., 2012). The term familiarity can also refer to subjective frequency—that is, the degree to which the printed word is familiar to the participant (e.g., Bird, Franklin, & Howard, 2001; Shao et al., 2013). Age of acquisition (AoA) has proven to be a very important parameter influencing lexical processing. Subjective (or rated) age-of-acquisition measures are collected in normative studies by asking adult participants to estimate the approximate age at which they believe that they learned the word. Objective age-of-acquisition measures are based on actually testing the ability of children of different ages to perform different lexical tasks. This is a more rare measure that is usually highly correlated with subjective age-of-acquisition ratings (Álvarez & Cuetos, 2007; Grigoriev & Oshhepkov, 2013; Johnston & Barry, 2006). A large number of studies have reported age-of-acquisition effects in different languages, populations, and experimental tasks (for a comprehensive review, see Johnston & Barry, 2006). The general pattern is that words that are learned earlier in life are processed faster and more accurately. Although some criticism has questioned the independence and validity of this measure (Bonin, Méot, Mermillod, Ferrand, & Barry, 2009; Zevin & Seidenberg, 2002), collecting age-of-acquisition ratings is still a gold standard in both noun and verb normative studies. Frequency usually refers to the number of occurrences of a word in a large corpus of written texts, although the word frequencies in spoken language may also be used, when available (e.g., in the CELEX database; Baayen, Piepenbrock, & Van Rijn, 1993). The impact of frequency on object-naming latencies is robust and has been shown in numerous studies (Alario et al., 2004; Barry et al., 1997; Cuetos et al., 1999; Ellis & Morrison, 1998): The more frequent the noun, the less time is needed to name the corresponding picture. However, the opposite pattern can be observed in an action-naming task: Szekely and colleagues (2005) found that high frequency was a predictor of shorter naming times for objects, but for action names the higherfrequency items took longer to produce. Szekely and colleagues explained this paradoxical result by the fact that participants use low-frequency “light verbs” for pictures that are difficult for them. Other authors have reported no

significant effect of frequency on action naming (Bonin et al., 2004; Cuetos & Alija, 2003; Schwitter, Boyer, Méot, Bonin, & Laganaro, 2004; Shao et al., 2013). Written frequency measures are usually strongly correlated with age-ofacquisition ratings, and it is controversial whether these two variables should be considered independently. Although some authors have shown that the age-of-acquisition effect on picture-naming speeds interacts with frequency (Barry et al., 1997), others have reported clear independent and determinant roles of both frequency and age of acquisition (e.g., Alario et al., 2004). In clinical populations, both the frequency effect (Cuetos et al., 2002; Nickels & Howard, 1995) and a reversed frequency effect (i.e., greater difficulties with the processing of more frequent words) have been observed in different tasks (Hoffman, Jefferies, & Ralph, 2011; Marshall, Pring, Chiat, & Robson, 2001). Length is a phonological factor that can be calculated as the number of syllables or the number of phonemes in a word. The common assumption is that during word production different segments of the word are sequentially encoded in the word frame, which leads to the prediction that longer words take more time to be encoded than shorter words (e.g., Alario et al., 2004). However, the experimental evidence for length effects is inconsistent. Some authors have reported no length effect at all, whereas other findings have shown an effect of length. For instance, Cuetos et al. (1999) found an independent syllabic length effect on object-naming latencies in Spanish: Reaction times were longer for longer words (a positive length effect). However, Alario et al. reported a marginal negative syllable-length effect for French: Longer nouns were produced more quickly than shorter nouns. The authors claimed that certain specific conditions should be met for the effect to be seen. For instance, Meyer, Roelofs, and Levelt (2003) revealed a syllable-length effect on production speed in a blocked presentation design, but not in a mixed design. Thus, another explanation is that the appearance of the length effect may depend on speakers’ response strategies, which may be revealed in some experimental designs but not others (Damian, Bowers, Stadthagen-Gonzalez, & Spalek, 2010; Meyer et al., 2003). In aphasia, a positive noun-length effect was also observed in patients with different diagnoses, including nonfluent, fluent, and apraxic individuals (Nickels & Howard, 1995). Argument structure, instrumentality, and name relation are parameters that are specific to verbs and are known to influence verb processing. Argument structure refers to the number of obligatory and optional participant roles required for a given verb (e.g., transitive verbs such as to beat require two arguments [“Peter is beating John”], whereas nontransitive verbs such as to run require only one argument [“Peter is running”]), and can be further defined in terms of qualitative categories. Studies that have focused on agrammatic aphasia in different languages have reported significant differences

Behav Res

between verb groups, the major tendency being toward increasing verb production difficulties with increasing numbers of arguments (Kim & Thompson, 2000, for English; Kiss, 2000, for Hungarian; Luzzatti et al., 2002, for Italian; Dragoy & Bastiaanse, 2010, for Russian; De Bleser & Kauschke, 2003, for German; see also Druks, 2002, for a review). Instrumentality is a conceptual factor that refers to an action’s conceptual representation containing an obligatory instrument/tool that is not a body part (e.g., to cut, to draw, in contrast to noninstrumental verbs such as to tear, to push). Name relation to a noun is a lexical–phonological parameter referring to the phonological similarity between an instrumental verb and its associated tool noun: for example, to bicycle is a name-related verb (to bicycle and a bicycle are homophonous). Jonkers and Bastiaanse (2007) found positive effects of instrumentality in a verb-naming task in a group of anomic aphasic speakers, and Malyutina, Iskra, and Sevan (2014) extended the findings for people with fluent and nonfluent aphasia and for elderly healthy individuals; instrumental verbs were better preserved than noninstrumental verbs. The results for name relation have been less consistent. Jonkers and Bastiaanse found a positive effect of name relation in a group of individuals with anomic aphasia, whereas Malyutina and colleagues demonstrated that verbs with a name relation to their associated noun were retrieved more poorly than those that were non-name-related. Databases with normative data about verbs and action pictures are by far less numerous than their noun/object counterparts. General information about existing verb and action databases, with their materials and parameters, is presented in Table 1. The aim of Fiez and Tranel (1997) was to develop a set of naming and recognition tests for the evaluation of lexical and conceptual processing of actions. For that purpose, norms for 280 English verbs and colored photographs of the corresponding actions were collected. The focus of Masterson and Druks (1998) was on making a list of English nouns and verbs and creating visual stimuli for them (164 objects and 102 actions) matched on a number of parameters, which would allow for the comparison of participants’ performance in experimental, assessment, and treatment tasks. Chiarello, Shears, and Lund (1999) examined dissociations between English verbs (n = 427), nouns (n = 555), and words of balanced verb–noun usage (n = 215), collecting, among several corpus-based statistical measures, imageability ratings for the words. The purpose of Bird, Franklin, and Howard (2001) was to collect imageability and age-of-acquisition ratings for a large set of words (n = 2,645), with 892 verbs and 213 function words among them, and in particular, to explore the relationship between a word’s class and the rated values. The interrelation of age of acquisition and several other parameters was also calculated. Cuetos and Alija (2003) offered a normative database for 100 Spanish verbs, using the visual materials from Druks and Masterson (2000).

Székely et al. (2004) performed the largest cross-linguistic project, which included databases of normative values for stimulus sets in seven languages (American English, German, Mexican Spanish, Italian, Bulgarian, Hungarian, and the Taiwan variant of Mandarin Chinese), the action and verb processing part of which represented by 275 items. A comprehensive normative database for French action photographs can be taken from Fiez and Tranel, and the corresponding verbs were presented in Bonin, Boyer, Méot, Fayol, and Droit (2004). Schwitter et al. (2004) also provided psycholinguistic norms for French, this time for drawings and not photographs, some of which (71 items) were taken from Druks and Masterson, and the rest (41 items) were originally developed. A comparison of two data sets for one language, based on photographs and black-and-white drawings, showed that when photographs were used, fewer items obtained %NA scores higher than or equal to 80 %, which allowed Schwitter et al. to recommend the use of drawings for research and clinical practice. Cameirão and Vicente (2010) described the collection of age-of-acquisition norms for 1,749 Portuguese words of different word classes, including 373 verbs, and they presented a database with these norms and several other psycholinguistic parameters. Finally, the recently published work of Shao, Roelofs, and Meyer (2013) introduced a normative database of 124 Dutch verbs and action pictures: 100 items from Druks and Masterson, and 24 items from Konopka and Meyer (2012). Until recently, no normative database with psycholinguistic parameters has existed for the Russian language, despite high demand. Russian is a major East Slavic language belonging to the Indo-European language family, and is the official language of the Russian Federation. Russian is also spoken in former USSR countries (Armenia, Azerbaijan, Belarus, Estonia, Georgia, Kazakhstan, Kyrgyzstan, Latvia, Lithuania, Moldova, Tajikistan, Turkmenistan, Ukraine, and Uzbekistan) and by emigrant communities in different parts of the world (Bulgaria, Canada, China, Croatia, Czech Republic, Finland, Germany, Greece, Israel, Mongolia, Mozambique, Norway, Paraguay, Poland, Romania, Serbia, Slovakia, Sweden, the United States, and Uruguay). Lewis, Simons, and Fennig (2013) estimated the total number of Russian speakers in the world to be 161,727,650, with an additional 110 million people speaking it as a second language. Russian has a rich derivational and inflectional morphology, uses the Cyrillic alphabet for writing, and features many other interesting characteristics that make it a valuable target for advanced (psycho)linguistic research. With the motivation to provide normative data for Russian, and thus to make it possible to control materials in Russian experimental research, several studies have been recently published. A study by Grigoriev, Baljasnikova, and Oshhepkov (2009) presented imageability data for 470 Russian nouns. In a subsequent study (Grigoriev,

✓

✓

✓

✓

Imag

Distribu-tional typicality

✓

✓

✓

✓

Objective AoA, objective visual complexity

✓

✓

✓

✓

Duration of depicted actions

✓

✓

✓

✓

112

Image variability

✓

✓

✓

✓

✓

✓

✓

✓

Concreteness, neighborhood density, length in letters, number of orthographic neighbors, number of phonological neighbors

✓

✓

✓

✓

✓

✓

✓

✓

✓

✓

✓

124

Druks & Masterson (2000; 100 items), Konopka & Meyer (2012; 24 items)

–

373

Dutch

Shao, Roelofs, & Meyer (2013)

Portuguese

Cameirão & Vicente (2010)

“✓” stands for parameters normed in the study or reported there on the basis of other sources. NA = name agreement (in terms of %NA, the H statistic, or both); IA = image agreement; Fam = conceptual familiarity; SCV = subjective visual complexity; Freq = frequency; AoA = age of acquisition; Imag = imageability; Length = word length in syllables or letters

Other parameters

Naming latencies Concrete-ness

✓

✓

AoA

Semantic categories for verbs and nouns

✓

✓

✓

✓

✓

Freq

Length

✓

✓

✓

✓

SVC

✓

✓ ✓

✓

142 ✓

✓

275 ✓

✓

100 ✓

Fam

892

Druks & Masterson (2000; 71 items), original additional drawings (41 item)

Fiez & Tranel (1997)

IA

427

French

Schwitter et al. (2004)

French

Bonin, Boyer, Méot, Fayol, & Droit (2004)

102

Druks & Masterson (2000)

English, Hungarian, Spanish, Italian, Bulgarian, and Chinese black-and-white line drawings

Székely et al. (2004)

✓

-

Spanish

Cuetos & Alija (2003)

280

-

English

Bird, Franklin, & Howard (2001)

✓

Black-and-white drawings

English

Chiarello, Shears, & Lund (1999)

NA

Colored photographs

Visual materials

English

Masterson & Druks (1998)

Number of items

English

Language(s)

Fiez & Tranel (1997)

Table 1 Normative studies and databases of verbs and actions for English, Spanish, Hungarian, Italian, Bulgarian, Chinese, French, Portuguese, and Dutch

Behav Res

Behav Res

Oshhepkov, Baljasnikova, & Orlova, 2010), data on name agreement (%NA and the H statistic), familiarity, and picture-name agreement were collected for 286 object pictures taken from Snodgrass and Vanderwart (1980), 90 of which were modified, and for eight additional pictures taken from Morrison, Chappell, and Ellis (1997). Finally, in a recently published article, Grigoriev and Oshhepkov (2013) reported objective age-of-acquisition measures for the 286 pictures from Grigoriev et al. (2010). In addition, another normative study for Russian was performed by a different research group (Tsaparina et al., 2011), which continued the collection of norms for object pictures from Snodgrass and Vanderwart and provided data on 260 Russian nouns and colorized images created by Rossion and Pourtois (2004), based on the drawings from Snodgrass and Vanderwart. The parameters included in the study were name agreement (%NA and the H statistic), image agreement, conceptual familiarity, imageability, age of acquisition, objective word frequency, and objective visual complexity. Grigoriev and Oshhepkov did not perform a direct comparison between their normative data and those reported by Tsaparina et al., except for the high correlation between objective age of acquisition obtained in their study and the rated age of acquisition from Tsaparina et al. Despite the emerging interest in normative data collection for nouns and object pictures in Russian, to our knowledge no available databases include Russian verbs and the corresponding action pictures, in addition to reporting reliable normative data collected from a considerable number of language speakers and taking into account other linguistic characteristics that are critical for verb processing. A recently published study by Pashneva (2013) presents name agreement norms and data on imageability, concept familiarity, and age of acquisition for 275 action pictures retrieved from the database of Bates et al. (2003). As a first stage, a picture study was performed in which the dominant names for action pictures were obtained. In the second stage, the normative data on imageability, concept familiarity, and age of acquisition were collected for the dominant names obtained in the first stage. Although this was the first attempt to create a verb and action stimulus database for Russian, the reliability of the results is disputable, due to the small number of participants in the first stage of the study (n = 34) and because the overall obtained name agreement scores were rather low (M = 55.18, SD = 23.37, for %NA; M = 1.89, SD = 0.92, for the H statistic). Our aim was to create a comprehensive verb–action picture normative database that would contain stimuli and the values of relevant (psycho)linguistic parameters that affect lexical processing. Given the lack of similar resources for Russian, our database will be useful not only for research purposes, but also for the development of speech-related diagnostic and therapeutic tools for Russian-speaking populations all over the world. The parameters included in the database are name

agreement (%NA and H statistics), objective and subjective visual complexity, image agreement, imageability, conceptual familiarity, age of acquisition, verb lemma frequency, number of arguments, length in syllables (third-person and infinitive forms), instrumentality, and name relatedness.

Method Stimuli The decision to create original visual stimuli was made because the number of depictable verbs that we wished to include in the study exceeded the number of items in the existing picture sets. The initial data set consisted of 414 depictable verbs and corresponding pictures of actions. For the pictures, we used 414 black-and-white drawings of actions specifically created by an artist for this project (the entire set is available online at http://neuroling.ru/en/db.htm ). Procedure Objective verb and action-picture parameters were obtained from available dictionaries or on the basis of the independent judgment of at least two professional linguists. If the judgments of the two experts differed, the case was discussed with a third expert, and a decision was made. Verb argument structure descriptions were taken from a dictionary (Ozhegov & Shvedova, 1992, available at http://dic. academic.ru/contents.nsf/ogegova/), with the number of arguments being defined as the number of obligatory arguments mentioned in the definition of the verb depicted in an image. Then the depiction of the verb was checked: If the argument was not present in the picture, it was not counted (e.g., the verb fotografirovat’ 1 “to take photos” may have a direct object, but it was not present in the picture, so the item was assigned “1” as the number of arguments, which referred to the participant present in the picture). If the argument of the verb could not be depicted because it was not a visible material object (e.g., pet’ “to sing”), the argument structure was assigned the value of “1/2.” It should be highlighted that the argument structure presented in the database is based on the concrete action depictions; hence, it is not recommended to use it as canonical argument structure for experiments that do not include those particular visual stimuli. Verb lemma frequency was extracted from Lyashevskaya and Sharov (2009), a Russian frequency dictionary based on the Russian National Corpus of written and spoken texts of various genres (the total size being 92 million words), available at http://dict. ruslang.ru/freq.php. The values were also transformed to 1 All of the examples are written in transliteration and accompanied with an English translation.

Behav Res

avoid data skewness using the formula log(1 + x), where x is the verb lemma frequency (Barry et al., 1997; Cuetos & Alija, 2003). The values of instrumentality and name relation were assigned by professional linguists. Since verb morphology is different from noun morphology in Russian, and full verb–noun homophony is not possible, the notion of name relation was broadened, so that a verb was considered name-related if it had the same root as the noun referring to the instrument/tool, but not if the verb and noun were completely homophonous, as in the case of English (e.g., a bicycle–to bicycle). Length was defined as the number of syllables in the form of the present tense third-person singular/plural, depending on the number of actors in the picture, and also as the number of syllables in the infinitive form. The objective visual complexity was defined as the file size in JPG format, in kilobytes, following the suggestion of Tsaparina et al. (2011). The normative data for the pictures (name agreement, action familiarity, and subjective visual complexity) and verbs (age of acquisition, imageability, and image agreement) were collected via the online survey platform http://virtualexs.ru/. The verbs were divided into six subsets, of 58 to 81 items each (mean = 74.7). Two lists were created for each subset, resulting in 12 lists overall. One list for each verb subset contained picture-based tasks—naming, familiarity, and subjective complexity judgment—whereas the second list contained word-based tasks—age of acquisition, imageability judgment, and image agreement. In the picture-based lists, for the collection of name agreement scores, the pictures (for an example, see Fig. 1) were displayed on a white background, and the participants were asked for a one-word answer to the question “What is the character (characters) doing in the picture?” This task resulted in the production of verbs in the third-person singular form, which has been proven to be more natural for Russian than the production of the infinitive form in an action-naming task (Akinina & Dragoy, 2012; Kozintseva et al., 2013). The participants were explicitly instructed not to give multiword expressions as answers. In the action familiarity task, the participants were asked to estimate how familiar the depicted action was in terms of how often they performed the action, observed others performing it, or thought about it, using the five-point scale from 1 = barely familiar to 5 = very familiar.2 The subjective visual complexity scores were collected by

2 The five-point scales used in this study were adapted from Snodgrass and Vanderwart (1980) and are widely used in noun- and object-norming studies, although some verb- and action-norming studies have been based on seven-point scales (e.g., Bird et al., 2001; Chiarello et al., 1999; Cuetos & Alija, 2003; Masterson & Druks, 1998; Schwitter et al., 2004; Shao et al., 2013). The normative data for Russian nouns and objects are collected using five-point scales to allow for a comparison of the present data for verbs and actions with future data for nouns and objects.

Fig. 1 Example visual stimulus, for the verb doit’ (“to milk”)

asking participants to evaluate the complexity of the picture, but not of the action itself, on the basis of the numbers of lines and details present in the picture, using a five-point scale from 1 = simple picture to 5 = complex picture. The norms in word-based lists were obtained on the basis of verbal modality. The verb in the infinitive form (e.g., doit’ “to milk”) was displayed on the screen. In the age-ofacquisition task, the participants were instructed to estimate the approximate age at which, in their opinion, they had learned the verb, on a five-point scale from 1 = 0–3 years to 5 = 12 years and later, with each point representing an interval of 3 years. The instructions for the imageability task were formulated as follows: The participants were asked to indicate how easy it was for them to imagine the action depicted in the picture, using a five-point scale from 1 = easy to imagine to 5 = hard to imagine (the modality of the images evoked by the verb—visual, auditory, tactile, etc.—was not specified). The imageability scale in this study was reversed relative to those in other verb-norming studies (Bird et al., 2001; Bonin et al., 2004; Chiarello et al., 1999; Cuetos & Alija, 2003; Masterson & Druks, 1998; Shao et al., 2013), where 1 has usually been used to designate the least-imageable verbs, and the opposite end of the scale corresponds to the most-imageable verbs. A reversed scale was used in the present study because it might better reflect the amount of effort needed to evoke mental images of an action, with smaller numbers indicating less effort, and greater numbers standing for more effort. In the image agreement task, the participants were asked to generate a mental image of the action corresponding to the displayed verb, then to proceed to the next slide featuring the picture (see

Behav Res

Fig. 1) developed for the verb, and to evaluate the degree of the match between the generated mental image and the presented picture, on a five-point scale from 1 = does not match at all to 5 = matches very well. Participants The links to the surveys were distributed online. Each participant received a link to no more than one list for a subset of the verbs (either a word-based list or a picture-based list, but not both), but participants were allowed to participate in multiple surveys, as long as each of them involved different verb subsets. For 11 of the lists, a total number of 100 participants’ responses were obtained, and for one word-based list, the responses of 102 participants were obtained; 1,202 surveys were completed, in total. In a demographic questionnaire preceding the main test, the participants reported themselves as being neurologically healthy native speakers of Russian (869 females, 332 males, with one of unidentified sex; age range: 16–76 years, M = 27.46, SD = 10.52; the level of education ranged from secondary school to a doctoral degree, with 85.19 % of the participants having at least some higher education).

The median of %NA was 80 %, a number that is often used as a threshold for selecting experimental and clinical materials (e.g., Cuetos & Alija, 2003). The mean of H = 1.20 and the positive skewness of this measure suggest relatively high name agreement scores for the data set. The statistics on subjective visual complexity indicate that the pictures were rated as having medium levels of complexity, on average (see the low interquartile range = 0.60 and the low skewness = 0.03), although the range of values (1.69–3.83) suggests that the whole scale was used. The scores of image agreement were high (M = 3.96) and positively skewed, meaning that the participants tended to estimate the pictures as matching with their generated mental images, although the whole scale was also used (range 1.89–4.96). The statistics on age of acquisition suggest that the verbs were generally rated as having been acquired early (M = 1.87, range 1.07–3.35). The imageability scores were low (M = 1.29), with a relatively narrow range (1– 2.57), indicating that few verbs in this data set are hard to picture in mind. The actions in the database were also rated as mostly familiar (M = 3.69) and the whole scale was used by the participants (range = 1.90–4.97). Correlation analysis

Results and discussion Normative data were collected for the total of 414 verbs and their pictorial counterparts. For 39 of the verbs, the obtained dominant (most frequent) name for the pictorial representation differed from the expected target name; in these cases, the data for pictures and for verbs were separated, regarded as incomplete, and excluded from further analysis, but they are provided in the supplementary materials of the main database. For the remaining 375 verbs, the means and standard deviations for the parameters of age of acquisition, familiarity, subjective complexity, imageability, and image agreement were calculated and entered into the database, along with the previously identified values for frequency, argument structure, instrumentality, name relation, and length (two measures). The distribution of different verbs used by participants to name each picture is also present in the database. The %NA values were estimated on the basis of the expected target name for each picture. The complete database is available for download in Excel format at http://neuroling.ru/en/db.htm. Descriptive statistics Descriptive statistics for the normative parameters (two measures of name agreement, subjective and objective visual complexity, image agreement, age of acquisition, imageability, familiarity, frequency, and length) are presented in Table 2. Table 3 contains the distribution of items for instrumentality, name relation, and argument structure.

The correlations among the obtained values of the psycholinguistic parameters show the interrelations among the measures. A pairwise two-tailed Pearson correlation analysis was performed on the following parameters: %NA, H, subjective and objective visual complexity, image agreement, age of acquisition, familiarity, log-transformed frequency, and length in form of third person (see Table 4). The correlations with length in the infinitive form are not reported, since the two length measures are highly correlated [r(373) = –.919, p < .001]. Table 5 presents the previously reported correlations among the normative parameters of the verbs and action pictures and their correspondence to the findings in the present study. The correlations obtained in the present study are generally consistent with those reported for other verb databases. Each of the significant correlations obtained in the present study is discussed below; the correlations that did not reach the level of significance are not mentioned, unless the observed lack of relationship between psycholinguistic parameters contradicts previous findings. As expected, a strong3 negative correlation [r(373) = –.932, p < .001] was found between %NA and H, which is due to the fact that the former is a component of the latter by definition: the value of %NA is a part of the H formula. The name agreement measures (both %NA and H) correlated with imageability [r(373) = –.218, p < .001; r(373) = .279, p < 3 In the use of effect size terms, we adhere to the conventions of Cohen (1988): “small” correlations = .1–.3, “moderate” = .3–.5, “large” > .5.

Behav Res Table 2 Descriptive statistics for 375 Russian verbs and action pictures

Mean Median SD Minimum value Maximum value 25th percentile 75th percentile Quartile range Skewness

%NA

H

SVC

OVC

IA

AoA

Imag

Fam

Freq

LogFreq

Length: Pers

Length: Inf

76.01 80.00 18.18 28.00 100.00 63.00 91.50 28.50 –0.72

1.20 1.07 0.76 0.00 3.66 0.59 1.70 1.11 0.63

2.69 2.70 0.43 1.69 3.83 2.39 2.98 0.60 0.03

264.28 237.00 113.68 93.30 844.00 185.50 313.50 128.00 1.26

3.96 4.15 0.71 1.89 4.96 3.43 4.55 1.12 –0.77

1.87 1.81 0.47 1.07 3.35 1.50 2.20 0.70 0.58

1.29 1.23 0.23 1.00 2.57 1.13 1.39 0.26 1.70

3.69 3.70 0.71 1.90 4.97 3.14 4.26 1.12 –0.14

45.27 8.70 108.66 0.00 957.10 3.30 32.05 28.75 4.67

1.12 0.99 0.64 0.00 2.98 0.63 1.52 0.89 0.67

3.22 3 1.23 1 6 2 4 2 0.37

2.66 2 0.98 1 6 2 3 1 0.60

%NA = percentage of name agreement; H = H statistic; SCV = subjective visual complexity; OVC = objective visual complexity; IA = image agreement; AoA = age of acquisition; Imag = imageability; Fam = familiarity; Freq = lemma frequency per million; LogFreq = lemma frequency per million, transformed according to the formula log(1 + x); Length: Pers = length in syllables in the form of third person; Length: Inf = length in the infinitive form

.001, respectively], which is consistent with the previous findings (Bonin et al., 2004; Shao et al., 2013): The opposite direction of the correlation is explained by the reverse normative scale used in the present study (where 1 = easy to imagine and 5 = hard to imagine, unlike in other studies, where 1 has referred to the least imageable point of the scale). The correlation between name agreement and imageability suggests that verbs that more easily evoke mental images tend to be named more uniformly and to obtain more responses that are the same as the target name. The name agreement measures also correlated with image agreement. For %NA, r(373) = .209, p < .001; these correlations were revealed by Bonin and colleagues (Bonin et al. 2004) and Shao and colleagues (2013), as well. For H statistics, r(373) = –.225, p < .001; the effect was also found in several studies (Bonin et al., 2004; Fiez & Tranel, 1997). The correlation between H and image agreement indicates that items that have a good match between the mental image and the picture are given more uniform names. These tendencies may occur due to the presence of a conventional mental image related to the verb. This conventional mental image may facilitate image generation in the imageability task. In turn, if a conventional image is represented in the picture, it may lead to consistent naming responses that more frequently coincide with the images the participants pictured in mind in the image agreement task.

Finally, the correlations between the name agreement measures (both %NA and H) and subjective visual complexity [r(373) = –.218, p < .001; r(373) = .259, p < .001, respectively] suggest that the pictures that were rated as being more complex received less uniform naming responses, and vice versa. Again, this may be due to the perceived visual simplicity of the conventional image (note that we found no correlation between the name agreement measures and objective visual complexity, meaning that those images were not objectively simpler, but they seemed simpler due to their conventionality). However, in other verb databases the correlations between the name agreement measures and subjective visual complexity did not reach the level of significance (Bonin et al., 2004; Cuetos & Alija, 2003; Fiez & Tranel, 1997; Schwitter et al., 2004; Shao et al., 2013). The correlation between subjective and objective visual complexity was strong and positive [r(373) = .534, p < .001], suggesting that objective measures of visual complexity can be used as an estimate of speakers’ perspectives on the amount of processing required for visual recognition of an action. Subjective visual complexity negatively correlated with familiarity [r(373) = –.317, p < .001]. The same correlation of subjective visual complexity with familiarity was found in the previous studies (see Bonin et al., 2004; Fiez &

Table 3 Argument structure, instrumentality, and name relations for 375 Russian verbs Instrumentality and Name Relation

Argument Structure

Verb Class

N of Verbs

Percentage

N of Arguments

N of Verbs

Percentage

Noninstrumental Instrumental Name-related Non-name-related

230 145 39 106

61.33 38.67 26.90 73.10

1 1/2 2 3

148 4 217 6

39.47 1.07 57.87 1.60

% % % %

% % % %

Behav Res Table 4 Correlation matrix of normative parameters for 375 Russian verbs and pictures of actions

H SVC OVC IA AoA Imag Fam LogFreq Length

%NA

H

SVC

OVC

IA

AoA

Imag

Fam

LogFreq

–.932* –.218* –.128 .209* .021 –.218* .076 .013 –.027

.259* .148 –.225* .002 .279* –.132 .001 –.003

.534* –.160 .162 .287* –.317* –.069 .071

–.076 .160 .083 –.178 –.154 .265*

.150 –.290* .120 –.253* .058

.427* –.341* –.396* .165

–.253* .072 –.055

.418* .026

–.363*

%NA = percentage of name agreement, H = H statistic, SVC = subjective visual complexity, OVC = objective visual complexity, IA = image agreement, AoA = age of acquisition, Imag = imageability, Fam = familiarity, LogFreq = log-transformed frequency, Length = length in the third-person form. The α was set to p < .001 (p < .05, Bonferroni corrected); significant correlations are marked with asterisks

Tranel, 1997), which indicates that more familiar actions are usually depicted with fewer details. The fact that correlations with familiarity and imageability [r(373) = .287, p < .001] were observed only for subjective visual complexity, and not for objective visual complexity, may imply that subjective ratings of visual complexity, imageability, and familiarity are biased by the same semantic properties of the concept. Besides, in our study the familiarity question preceded the visual complexity question, which may also have caused a priming effect for the visual complexity task (but note that the imageability task was in another experimental list). Therefore, it is not clear whether subjective visual complexity is an independent parameter, or which visual complexity measure—subjective or objective—reflects the real visualprocessing effort. A regression study with a behavioral dependent variable such as action-naming reaction times could clarify this issue. As expected, age of acquisition correlated significantly with imageability, familiarity, and frequency, which means that verbs that refer to actions that are more frequent, imageable, and familiar tend to be rated as having been learned earlier. This result is highly consistent and has been replicated in almost every norming study of verbs, if age of acquisition was considered (Bird et al., 2001; Bonin et al., 2004; Cameirão & Vicente, 2010; Cuetos & Alija, 2003; Schwitter et al., 2004; Shao et al., 2013). The opposite (positive) correlation between age of acquisition and imageability in the present study is explained by the use of the reverse imageability scale. However, a very persistent correlation between age of acquisition and length (Bird et al., 2001; Cameirão & Vicente, 2010; Cuetos & Alija, 2003; Shao et al., 2013) was not maintained in the present study, although it approached significance. For imageability, correlations with subjective visual complexity [positive, r(373) = .287, p < .001], image agreement

[negative, r(373) = –.290, p < .001], and familiarity [negative, r(373) = –.253, p < .001] were observed. The positive correlation between imageability and subjective visual complexity suggests that verbs that require more effort to generate a mental image for are also related to relatively more complex pictures in our database. This reflects the expected link between the difficulty of generating an image and the objective complexity of a drawing. Similarly, the negative correlation between imageability and image agreement indicates that verbs that require more effort to generate a mental image for are naturally related to pictures that are nonpreferred matches for the images generated by the participants. We assume that these observations pertain to the absence of a conventional verb-related image. Imageability and familiarity are negatively correlated, which indicates that the actions that are rated as more familiar tend to be related to verbs that prompt easier generation of mental images. Familiarity was shown to be negatively correlated with subjective visual complexity [r(373) = –.317, p < .001], age of acquisition [r(373) = –.341, p < .001], and imageability [r(373) = –.253, p < .001]. A positive correlation with logtransformed frequency [r(373) = .418, p < .001] was observed. The moderate negative correlation between familiarity and subjective visual complexity indicates that the pictures with more familiar actions are rated as being less complex, and vice versa. This result was also found in other verb databases (Bonin et al., 2004; Fiez & Tranel, 1997). The moderate correlation between familiarity and logtransformed frequency is in line with previous research (Bird et al., 2001; Bonin et al., 2004; Cuetos & Alija, 2003; Schwitter et al., 2004) and suggests that more frequently used verbs are related to more familiar actions. The correlation between familiarity and image agreement was found in the present study for the first time, and indicates that for more familiar actions, the

Behav Res Table 5 Correlations reported in previous verb norming studies Present Fiez & Chiarello, Bird, Franklin, Cuetos & Bonin, Boyer, Schwitter Cameirão Shao, Roelofs, Study Tranel (1997) Shears, & & Howard Alija (2003) Méot, Fayol, et al. (2004) & Vicente & Meyer (2013) Lund (1999) (2001) & Droit (2004) (2010) %NA–H %NA–SVC %NA–IA %NA–AoA %NA–Imag %NA–Fam %NA–Freq %NA–Length H–SVC

– – + n.s. – n.s. n.s. n.s. +

H–IA H–AoA H–Imag H–Fam H–Freq H–Length SVC–IA SVC–AoA SVC–Imag SVC–Fam SVC–Freq SVC–Length IA–AoA IA–Imag IA–Fam IA–Freq IA–Length AoA–Imag

– n.s. + n.s. n.s. n.s. n.s. n.s. + – n.s. n.s. n.s. – + – n.s. +

AoA–Fam AoA–Freq AoA–Length Imag–Fam Imag–Freq Imag–Length Fam–Freq Fam–Length Freq–Length

– – n.s. – n.s. n.s. + n.s. –

–* n.s. +* – +* n.s. + n.s.

n.s.

–* n.s. –* n.s. n.s. n.s.

– – –

+ n.s. n.s. n.s. n.s.

–* –

n.s. n.s.

n.s. n.s. n.s. –* –

–*

–*

– –* +* n.s. + – +* – –*

– –* +* n.s. n.s. n.s. +* n.s.

– –*

*

n.s. n.s. n.s.

n.s. n.s. n.s.

–* + –* n.s. n.s.

–* *

n.s. n.s. n.s.

n.s.

+* n.s. n.s. –*

n.s. –*

–* n.s. +* – +*

*

n.s. + –*

n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s. n.s.

– n.s. n.s. +*

–* – –* n.s. *

n.s. n.s. +*

n.s. n.s. n.s. + –*

+* n.s. n.s.

+ –* +* – n.s. – n.s. n.s. –*

n.s. n.s. –* –* +* n.s. n.s.

–*

%NA = percentage of name agreement, H = H statistic, SVC = subjective visual complexity, IA = image agreement, AoA = age of acquisition, Imag = imageability, Fam = conceptual familiarity, Freq = frequency. The Length parameter in the present table refers to the length in syllables. In Bird et al. (2001), Fam was not specified for the word class. Schwitter et al. (2004) did not explicitly specify whether Freq was calculated for the lemma or the infinitive form. All correlations in the table have p < .05 or less; only correlations among variables that were included in the present study are reported. “+ ” means a positive correlation, “–” means a negative correlation, “n.s.” means that the correlation did not reach the level of significance, and a blank cell means that the variable or correlation was not analyzed. Significant correlations that are the same as in the present study (including those with correction for the inverse imageability scale in the present study) are marked with an asterisk

image offered to the participants tended to be scored as matching with the mental image—again, this may have been due to the presence of a conventional image.

The replication of the general correlation patterns between the variables confirms the reliability of the normative values obtained in the present study.

Behav Res

Analysis of name disagreement sources One of the most common research and clinical uses of the developed experimental stimuli involves a naming task, so it is important to analyze the normative variability in picture naming and its possible sources. In the present database, 49 % of items (182 out of 375) obtained name agreement scores lower than 80 %. These pictures were classified as either having multiple names (i.e., alternative names that could be considered correct, although they do not coincide with the target name) or erroneous names. In this study, the erroneous answers were defined as having the same semantic-category coordinates (e.g., chikhaet “sneezes” instead of kashljaet “coughs”) or responses that were not in any way connected to the target verb and that were probably due to action recognition failures (e.g., topit “stokes” instead of pechjot “bakes”). The ratio of pictures that had erroneous names among the answers was relatively low: 107 out of 182 items with %NA < 80 obtained multiple names without any erroneous names; for the other 75 images erroneous answers did occur, and the percentage of erroneous responses was low (range = 1–39, M = 6.26, SD = 7.81, median = 3, Q3 = 6). Table 6 presents a comparison between our data and the available data on the percentages of name agreement and the H statistics from other databases (Cuetos & Alija, 2003; Fiez & Tranel, 1997; Shao et al., 2013). The differences from the database of Fiez and Tranel on both measures did not reach significance, but name agreement in the present database was found to be significantly lower than in the findings of Cuetos and Alija and Shao et al. For %NA, the results of the independent-samples t test were t(473) = 3,081, p < .01, and t(196) = 6.092, p < .01, respectively (for the last one, Levene’s test was significant, so the degrees of freedom were adjusted from 477 to 196). For the H statistics, the results were t(473) = –5.410, p < .01, and t(243) = –9.881, p < .01, respectively (for the last one, Levene’s test was significant, so the degrees of freedom were adjusted from 477 to 243). There are several possible explanations for these discrepancies. First, the original set of the present study (375 items), as well as the data set of Fiez and Tranel (280 items), was more than twice as large as the sets of Cuetos and Alija (100 items) and Shao et al. (124

items). The expansion of the lists of verbs and actions beyond the most basic and simple items most likely led to the inclusion of complex concepts that subsequently entailed different interpretations when depicted, such as a change of perspective (e.g., gonit “chases” and ubegaet “runs away”). It should also be noted that the latter two databases profited from a preliminary filtering of the stimuli. That is, pictures that were in some way ambiguous and could prompt multiple names were excluded from the original set of Druks and Masterson (2000), used in those two studies. In addition, specific features of the Russian language discussed below (prefixes and regular morphological derivation patterns) could add to a wider naming variability. The cross-linguistic comparison of noun and object stimuli reported in Tsaparina et al. (2011) also revealed lower scores on %NA and higher scores on H for Russian, and the authors claimed that certain aspects of Russian morphology (e.g., diminutives, in the case of nouns) are at least partially responsible for the higher naming disagreement. The qualitative analysis of nontarget responses that were not visual recognition errors showed that they fell into the categories defined by Snodgrass and Vanderwart (1980) for nouns and into additional, verb-specific categories introduced by Fiez and Tranel (1997). In addition to naming failures (i.e., no response), those categories were synonyms (i.e., verbs that can be used interchangeably to refer to the depicted actions; e.g., kushaet instead of est, both meaning “eat”), coordinates (different exemplars of the same category; e.g., shtopaet “darns” instead of vyshivaet “embroiders”), superordinates (e.g., chinit “repairs” instead of nastraivaet “tunes up”), and subordinates (khokhochet “guffaws” instead of smejutsa “[they] laugh”). The verb-specific nontarget responses defined by Fiez and Tranel included changes of perspective (e.g., pugaetsa “is frightened” instead of pugaet “frightens”), part/whole relationships between actions (e.g., namylivaet “soaps” instead of kupaet “bathes tr”), and entailed actions (e.g., vonyaet “stinks” instead of kurit “smokes”). Figure 2 presents histograms for percentages of name agreement and H statistics. As we have already mentioned, the rich derivational morphology of the Russian language largely contributed to the

Table 6 Mean values and SDs from four verb normative databases

Language %NA, mean %NA, SD H, mean H, SD

Present Study

Fiez & Tranel (1997)

Cuetos & Alija (2003)

Shao et al. (2013)

Russian 76.01 18.18 1.20 0.76

English 73.64 24.05 1.15 0.87

Spanish 82.60 21.85 0.74 0.80

Dutch 86.62 14.95 0.57 0.51

Behav Res

Fig. 2 Histograms for %NA and H

higher rate of naming disagreement. For instance, for the target verb vyzhimat’ “to wring out,” the second most popular name after the target name vyzhimaet (54 % of responses) was otzhimaet “wrings out” (31 %), with the two prefixes being synonymous in this context. Another example of derivation contributing to naming disagreement would be when semantically specific names with prefixes were given instead of the more general names without prefixes, and the other way around, which can be regarded as subordinate and superordinate responses, respectively (for instance, kopaet “digs” [78 % of responses] vs. vskapyvaet “digs up” [22 % of responses], or vyplyvaet “sails out” [53 % of responses] vs. plyvjot “sails” [34 % of responses]). Out of 375 pictures, 155 (41.33 %) were assigned at least one nontarget name that could be regarded as the correct name and that was derived from the same root by morphological means involving a prefix addition (or prefix change or elimination, for verbs that initially had one). For example, for the target verb vstajot “rises,” the nontarget name privstajot “half-rises” can be taken as correct, whereas the name saditsa “sits down” is erroneous. For the 155 verbs, such prefix-based alternatives constituted 23.34 % of the nontarget alternative names, and 41.64 % of all nontarget responses. For eight items, this was the only source of naming variability. When all 375 verbs were taken into account, prefix-based alterations constituted 19.16 % of all nontarget responses. Finally, in some cases a change of perspective was implemented through a regular morphological derivation that changed the verb argument structure—for example, zapravlyaet “fills up the tank” and zapravlyaetsa “fills up one’s tank,” or cheshetsa “scratches oneself” and cheshet “scratches.” The results of the name disagreement analysis have considerable implications for the use of the presented database for experimental and clinical purposes. Most researchers perform trimming procedures on the materials to be used; for example, they may use a threshold for %NA or H in

order for items to be included in a specific stimulus set. However, the specificity of action-naming variance enhanced by Russian morphology warrants other approaches in the scoring of the naming task. As a possible solution, Fiez and Tranel (1997) developed and tested administration and scoring procedures to deal with a similar variance in their database. These included cueing (pointing to some portions of the photograph in order to highlight a certain component of an action or to provoke a change of perspective) and prompting the participants for additional responses during the naming task. The scoring procedure allowed the response to be counted as correct if it was elicited from more than 5 % of the participants or if it was judged to be an accurate and specific name for the depicted action. Such possibilities must be taken into consideration in research or during the development of naming assessment and treatment tools using the presented Russian-language database of verbs and action pictures.

Conclusion The present study provides norms for a set of 375 action pictures and verbs in the Russian language. The parameters presented in the database are name agreement (percentage of name agreement and H statistic), objective and subjective visual complexity, image agreement, imageability, familiarity, age of acquisition, verb lemma frequency, number of arguments present in the picture, word length in syllables (in the third-person and infinitive forms), instrumentality, and name relatedness. The results of the correlation analysis of these parameters are highly consistent with those reported in other databases, which confirms the reliability of the collected Russian normative data. The scores of the name agreement and the H statistics were compared to those from other verb

Behav Res

databases. Possible explanations for the finding of higher naming disagreement in Russian are related to its rich derivational morphology, among other factors. Russian-specific features of the developed database also entail particular recommendations for further clinical and research use of the presented stimuli. The full database is available online and can be freely downloaded from http://neuroling.ru/en/db.htm. Author Note This article is an output of a research project implemented as part of the Basic Research Program at the National Research University Higher School of Economics (HSE). We thank Victor M. Shklovsky, who inspired the authors to implement the project. We are also grateful to all of the participants for their contributions to this study, Kelly Callahan for proofreading the article, and those of our colleagues who are not mentioned among the authors for their help and advice.

References Akinina, Y., & Dragoy, O. (2012). Semantic and phonological naming therapy: New criteria of therapy effects assessment. Stem-, Spraaken Taalpathologie, 17, 15–17. Alario, F. X., & Ferrand, L. (1999). A set of 400 pictures standardized for French: Norms for name agreement, image agreement, familiarity, visual complexity, image variability, and age of acquisition. Behavior Research Methods, Instruments, & Computers, 31, 531– 552. doi:10.3758/BF03200732 Alario, F. X., Ferrand, L., Laganaro, M., New, B., Frauenfelder, U. H., & Segui, J. (2004). Predictors of picture naming speed. Behavior Research Methods, Instruments, & Computers, 36, 140–155. doi: 10.3758/BF03195559 Álvarez, B., & Cuetos, F. (2007). Objective age of acquisition norms for a set of 328 words in Spanish. Behavior Research Methods, 39, 377– 383. doi:10.3758/BF03193006 Baayen, R. H., Piepenbrock, R., & Van Rijn, H. (1993). The CELEX lexical database (CD-ROM). Philadelphia, PA: Linguistic Data Consortium, University of Pennsylvania. Barry, C., Morrison, C. M., & Ellis, A. W. (1997). Naming the Snodgrass and Vanderwart pictures: Effects of age of acquisition, frequency, and name agreement. Quarterly Journal of Experimental Psychology, 50A, 560–585. doi:10.1080/783663595 Bastiaanse, R. (1991). Retrieval of instrumental verbs in aphasia: An explorative study. Clinical Linguistics and Phonetics, 5, 355–368. Bates, E., D’Amico, S., Jacobsen, T., Székely, A., Andonova, E., Devescovi, A., … Tzeng, O. (2003). Timed picture naming in seven languages. Psychonomic Bulletin & Review, 10, 344–380. doi:10. 3758/BF03196494 Berman, S., Friedman, D., Hamberger, M., & Snodgrass, J. G. (1989). Developmental picture norms: Relationships between name agreement, familiarity, and visual complexity for child and adult ratings of two sets of line drawings. Behavior Research Methods, Instruments, & Computers, 21, 371–382. doi:10.3758/BF03202800 Bird, H., Franklin, S., & Howard, D. (2001). Age of acquisition and imageability ratings for a large set of words, including verbs and function words. Behavior Research Methods, Instruments, & Computers, 33, 73–79. doi:10.3758/BF03195349 Bird, H., Howard, D., & Franklin, S. (2003). Verbs and nouns: The importance of being imageable. Journal of Neurolinguistics, 16, 113–149. Bonin, P., Boyer, B., Méot, A., Fayol, M., & Droit, S. (2004). Psycholinguistic norms for action photographs in French and their

relationships with spoken and written latencies. Behavior Research Methods, Instruments, & Computers, 36, 127–139. doi:10.3758/ BF03195558 Bonin, P., Méot, A., Mermillod, M., Ferrand, L., & Barry, C. (2009). The effects of age of acquisition and frequency trajectory on object naming: Comments on Pérez (2007). Quarterly Journal of Experimental Psychology, 2, 1132–1140. doi:10.1080/ 17470210802618850 Bonin, P., Peereman, R., Malardier, N., Méot, A., & Chalard, M. (2003). A new set of 299 pictures for psycholinguistic studies: French norms for name agreement, image agreement, conceptual familiarity, visual complexity, image variability, age of acquisition, and naming latencies. Behavior Research Methods, Instruments, & Computers, 35, 158–167. doi:10.3758/BF03195507 Cameirão, M. L., & Vicente, S. G. (2010). Age-of-acquisition norms for a set of 1,749 Portuguese words. Behavior Research Methods, 42, 474–480. doi:10.3758/BRM.42.2.474 Cappa, S. F., & Perani, D. (2003). The neural correlates of noun and verb processing. Journal of Neurolinguistics, 16, 183–189. Chiarello, C., Shears, C., & Lund, K. (1999). Imageability and distributional typicality measures of nouns and verbs in contemporary English. Behavior Research Methods, Instruments, & Computers, 31, 603–637. doi:10.3758/BF03200739 Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Connell, L., & Lynott, D. (2012). Strength of perceptual experience predicts word processing performance better than concreteness or imageability. Cognition, 125, 452–465. doi:10.1016/j.cognition. 2012.07.010 Cortese, M. J., Simpson, G. B., & Woolsey, S. (1997). Effects of association and imageability on phonological mapping. Psychonomic Bulletin & Review, 4, 226–231. doi:10.3758/BF03209397 Cotelli, M., Borroni, B., Manenti, R., Alberici, A., Calabria, M., Agosti, C., … Cappa, S. F. (2006). Action and object naming in frontotemporal dementia, progressive supranuclear palsy, and corticobasal degeneration. Neuropsychology, 20, 558–565. Crepaldi, D., Berlingeri, M., Paulesu, E., & Luzzatti, C. (2011). A place for nouns and a place for verbs? A critical review of neurocognitive data on grammatical-class effects. Brain and Language, 116, 33–49. doi:10.1016/j.bandl.2010.09.005 Cuetos, F., Aguado, G., Izura, C., & Ellis, A. W. (2002). Aphasic naming in Spanish: Predictors and errors. Brain and Language, 82, 344– 365. doi:10.1016/S0093-934X(02)00038-X Cuetos, F., & Alija, M. (2003). Normative data and naming times for action pictures. Behavior Research Methods, Instruments, & Computers, 35, 168–177. doi:10.3758/BF03195508 Cuetos, F., Ellis, A. W., & Alvarez, B. (1999). Naming times for the Snodgrass and Vanderwart pictures in Spanish. Behavior Research Methods, Instruments, & Computers, 31, 650–658. doi:10.3758/ BF03200741 Cuetos, F., Samartino, T., & Ellis, A. W. (2012). Age of acquisition norms from elderly Spanish people: Characteristics and the prediction of word recognition performance in Alzheimer’s disease. Psicológica, 33, 59–76. Cycowicz, Y. M., Friedman, D., Rothstein, M., & Snodgrass, J. G. (1997). Picture naming by young children: Norms for name agreement, familiarity, and visual complexity. Journal of Experimental Child Psychology, 65, 171–237. doi:10.1006/jecp.1996.2356 d’Honincthun, P., & Pillon, A. (2008). Verb comprehension and naming in frontotemporal degeneration: The role of the static depiction of actions. Cortex, 44, 834–847. doi:10.1016/j.cortex.2007.04.003 Damian, M. F., Bowers, J. S., Stadthagen-Gonzalez, H., & Spalek, K. (2010). Does word length affect speech onset latencies when producing single words? Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 892–905. doi:10.1037/ a0019446

Behav Res De Bleser, R., & Kauschke, C. (2003). Acquisition and loss of nouns and verbs: Parallel or divergent patterns? Journal of Neurolinguistics, 16, 213–229. Dell’Acqua, R., Lotto, L., & Job, R. (2000). Naming times and standardized norms for the Italian PD/DPSS set of 266 pictures: Direct comparisons with American, English, French, and Spanish published databases. Behavior Research Methods, Instruments, & Computers, 32, 588–615. doi:10.3758/BF03200832 Dimitropoulou, M., Duñabeitia, J. A., Blitsas, P., & Carreiras, M. (2009). A standardized set of 260 pictures for Modern Greek: Norms for name agreement, age of acquisition, and visual complexity. Behavior Research Methods, 41, 584–589. doi:10.3758/BRM.41. 2.584 Dragoy, O., & Bastiaanse, R. (2010). Verb production and word order in Russian agrammatic speakers. Aphasiology, 24, 28–55. doi:10. 1080/02687030802586902 Druks, J. (2002). Verbs and nouns: Review of the literature. Journal of Neurolinguistics, 15, 289–315. Druks, J., & Masterson, J. (2000). An object and action naming battery. Hove, UK: Psychology Press. Ellis, A. W., & Morrison, C. M. (1998). Real age-of-acquisition effects in lexical retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition, 24, 515–523. doi:10.1037/0278-7393.24. 2.515 Fiez, J. A., & Tranel, D. (1997). Standardized stimuli and procedures for investigating the retrieval of lexical and conceptual knowledge for actions. Memory & Cognition, 25, 543–569. doi:10.3758/ BF03201129 Goodglass, H. (1993). Understanding aphasia. New York, NY: Academic Press. Grigoriev, A. A., Baljasnikova, O. V., & Oshhepkov, I. V. (2009). Ocenki obraznosti nazvanijj risunkov iz standartnogo nabora izobrazhenijj i rjada drugikh slov. Voprosy Psikholingvistiki, 9, 128–133. Grigoriev, A., & Oshhepkov, I. (2013). Objective age of acquisition norms for a set of 286 words in Russian: Relationships with other psycholinguistic variables. Behavior Research Methods, 45, 1208– 1217. doi:10.3758/s13428-013-0319-0 Grigoriev, A. A., Oshhepkov, I. V., Baljasnikova, O. V., & Orlova, E. A. (2010). Normativnye dannye po obraznosti, soglasovannosti naimenovanijj, znakomosti i sootvetstviju naimenovanija izobrazheniju dlja nabora iz 286 stimulov. Vestnik Moskovskogo gosudarstvennogo oblastnogo universiteta. Psikhologicheskie Nauki, 1, 42–51. Himmanen, S. A., Gentles, K., & Sailor, K. (2003). Rated familiarity, visual complexity, and image agreement and their relation to naming difficulty for items from the Boston Naming Test. Journal of Clinical and Experimental Neuropsychology, 25, 1178–1185. Hoffman, P., Jefferies, E., & Ralph, M. A. L. (2011). Remembering “zeal” but not “thing”: Reverse frequency effects as a consequence of deregulated semantic processing. Neuropsychologia, 49, 580–584. doi:10.1016/j.neuropsychologia.2010.12.036 Johnston, R. A., & Barry, C. (2006). Age of acquisition and lexical processing. Visual Cognition, 13, 789–845. doi:10.1080/ 13506280544000066 Jonkers, R., & Bastiaanse, R. (2007). Action naming in anomic aphasic speakers: Effects of instrumentality and name relation. Brain and Language, 102, 262–272. Kemmerer, D., & Tranel, D. (2000). Verb retrieval in brain-damaged subjects: 1. Analysis of stimulus, lexical, and conceptual factors. Brain and Language, 73, 347–392. Kim, M., & Thompson, C. K. (2000). Patterns of comprehension and production of nouns and verbs in agrammatism: Implications for lexical organization. Brain and Language, 74, 1–25. Kiss, K. (2000). Effect of verb complexity on agrammatic aphasics’ sentence production. In R. Bastiaanse & Y. Grodzinsky (Eds.),

Grammatical disorders in aphasia: A neurolinguistic perspective (pp. 152–170). London, UK: Whurr. Konopka, A., & Meyer, A. S. (2012). Priming sentence planning: The role of words and structures. Unpublished manuscript. Kozintseva, E. G., Dragoy, O., Malyutina, S., Ivanova, M. V., Sevan, D., Kuptsova, S., … Gutyrchik, E. F. (2013). Naming actions in nonfluent aphasia: An fMRI study of compensatory reorganization of brain activity. Stem-, Spraak- en Taalpathologie, 18, 98–99. Kremin, H., Akhutina, T., Basso, A., Davidoff, J., De Wilde, M., Kitzing, P., … Weniger, D. (2003). A cross-linguistic data bank for oral picture naming in Dutch, English, German, French, Italian, Russian, Spanish, and Swedish (PEDOI). Brain & Cognition, 53, 243–246. Lachman, R. (1973). Uncertainty effects on time to access the internal lexicon. Journal of Experimental Psychology, 99, 199–208. Lewis, M. P., Simons, G. F., & Fennig, C. D. (Eds.). (2013). Ethnologue: Languages of the world (17th ed.). Dallas, TX: SIL International. Available at www.ethnologue.com Luzzatti, C., Aggujaro, S., & Crepaldi, D. (2006). Verb–noun double dissociation in aphasia: Theoretical and neuroanatomical foundations. Cortex, 42, 875–883. Luzzatti, C., Raggi, R., Zonca, G., Pistarini, C., Contardi, A., & Pinna, G. D. (2002). Verb–noun double dissociation in aphasic lexical impairments: The role of word frequency and imageability. Brain and Language, 81, 432–444. Lyashevskaya, O. N., & Sharov, S. A. (2009). Chastotnyj slovar’ sovremennogo russkogo jazyka (na materialakh Natsional’nogo korpusa russkogo jazyka) [Frequency vocabulary of Modern Russian (based on the materials of Russian National Corpus]. Moscow, Russia: Azbukovnik. Retrieved from http://dict.ruslang. ru/freq.php Malyutina, S., Iskra, E., & Sevan, D. (2014). Aphasiology the effects of instrumentality and name relation on action naming in Russian speakers with aphasia, (May), 37–41. doi:10.1080/02687038.2014. 910589. Manoiloff, L., Artstein, M., Bélen Canavoso, M., Fernández, L., & Segui, J. (2010). Expanded norms for 400 experimental pictures in an Argentinean Spanish-speaking population. Behavior Research Methods, 42, 452–460. doi:10.3758/BRM.42.2.452 Marshall, J., Pring, T., Chiat, S., & Robson, J. (2001). When ottoman is easier than chair: An inverse frequency effect in jargon aphasia. Cortex, 37, 33–53. doi:10.1016/S0010-9452(08)70556-2 Martein, R. (1995). Norms for name and concept agreement, familiarity, visual complexity and image agreement on a set of 216 pictures. Psychologica Belgica, 35, 205–225. Masterson, J., & Druks, J. (1998). Description of a set of 164 nouns and 102 verbs matched for printed word frequency, familiarity and ageof-acquisition. Journal of Neurolinguistics, 11, 331–354. doi:10. 1016/S0911-6044(98)00023-2 Mätzig, S., Druks, J., Masterson, J., & Vigliocco, G. (2009). Noun and verb differences in picture naming: Past studies and new evidence. Cortex, 45, 738–758. doi:10.1016/j.cortex.2008.10.003 Meyer, A. S., Roelofs, A., & Levelt, W. J. M. (2003). Word length effects in object naming: The role of a response criterion. Journal of Memory and Language, 48, 131–147. doi:10.1016/S0749596X(02)00509-0 Morrison, C. M., Chappell, T. D., & Ellis, A. W. (1997). Age of acquisition norms for a large set of object names and their relation to adult estimates and other variables. Quarterly Journal of Experimental Psychology, 50A, 528–559. doi:10.1080/027249897392017 Nickels, L., & Howard, D. (1995). Aphasic naming: What matters? Neuropsychologia, 33, 1281–1303. doi:10.1016/0028-3932(95) 00102-9 Nishimoto, T., Miyawaki, K., Ueda, T., Une, Y., & Takahashi, M. (2005). Japanese normative set of 359 pictures. Behavior Research Methods, 37, 398–416. doi:10.3758/BF03192709

Behav Res Nishimoto, T., Ueda, T., Miyawaki, K., Une, Y., & Takahashi, M. (2012). The role of imagery-related properties in picture naming: A newly standardized set of 360 pictures for Japanese. Behavior Research Methods, 44, 934–945. doi:10.3758/ s13428-011-0176-7 Nisi, M., Longoni, A. M., & Snodgrass, J. G. (2000). Misure italiane per l’accordo sul nome, familiarità ed età di acquisizione, per le 260 figure di Snodgrass e Vanderwart (1980). Giornale Italiano di Psicologia, 27, 205–220. doi:10.1421/288 Ozhegov, S., & Shvedova, N. (1992). Tolkovy slovar’ russkogo yazyka [Russian explanatory dictionary]. Moscow, Russia: Az. Pashneva, S. (2013). Russkie normativnye imena dlja 275 izobrazhenij dejstvij I dannyje po ikh obraznosti, znakomosti i vozrastu usvojenija. Molodoj Uchjonyj, 10, 606–610. Pind, J., Jónsdóttir, H., Gissurardóttir, H., & Jónsson, F. (2000). Icelandic norms for the Snodgrass and Vanderwart (1980) pictures: Name and image agreement, familiarity, and age of acquisition. Scandinavian Journal of Psychology, 41, 41–48. doi:10.1111/1467-9450.00169 Rossion, B., & Pourtois, G. (2004). Revisiting Snodgrass and Vanderwart’s object pictorial set: The role of surface detail in basic-level object recognition. Perception, 33, 217–236. doi:10. 1068/p5117 Salmon, J. P., McMullen, P. A., & Filliter, J. H. (2010). Norms for two types of manipulability (graspability and functional usage), familiarity, and age of acquisition for 320 photographs of objects. Behavior Research Methods, 42, 82–95. doi:10.3758/BRM.42.1.82 Sanfeliu, M. C., & Fernandez, A. (1996). A set of 254 Snodgrass– Vanderwart pictures standardized for Spanish: Norms for name agreement, image agreement, familiarity, and visual complexity. Behavior Research Methods, Instruments, & Computers, 28, 537– 555. doi:10.3758/BF03200541 Schröder, A., Gemballa, T., Ruppin, S., & Wartenburger, I. (2012). German norms for semantic typicality, age of acquisition, and concept familiarity. Behavior Research Methods, 44, 380–394. doi:10. 3758/s13428-011-0164-y Schwitter, V., Boyer, B., Méot, A., Bonin, P., & Laganaro, M. (2004). French normative data and naming times for action pictures.

Behavior Research Methods, Instruments, & Computers, 36, 564– 576. doi:10.3758/BF03195603 Shao, Z., Roelofs, A., & Meyer, A. S. (2013). Predicting naming latencies for action pictures: Dutch norms. Behavior Research Methods, 46, 274–283. doi:10.3758/s13428-013-0358-6 Snodgrass, J. G., & Vanderwart, M. (1980). A standardized set of 260 pictures: Norms for name agreement, image agreement, familiarity, and visual complexity. Journal of Experimental Psychology: Human Learning and Memory, 6, 174–215. doi:10.1037/02787393.6.2.174 Snodgrass, J. G., & Yuditsky, T. (1996). Naming times for the Snodgrass and Vanderwart pictures. Behavior Research Methods, Instruments, & Computers, 28, 516–536. doi:10.3758/BF03200540 Szekely, A., & Bates, E. (2000). Objective visual complexity as a variable in studies of picture naming. CRL Newsletter, 12(2), 1–33. Szekely, A., D’Amico, S., Devescovi, A., Federmeier, K., Herron, D., Iyer, G., … Bates, E. (2005). Timed action and object naming. Cortex, 41, 7–25. doi:10.1016/S0010-9452(08)70174-6 Székely, A., Jacobsen, T., D’Amico, S., Devescovi, A., Andonova, E., Herron, D., … Bates, E. (2004). A new on-line resource for psycholinguistic studies. Journal of Memory and Language, 51, 247– 250. doi:10.1016/j.jml.2004.03.002 Tsaparina, D., Bonin, P., & Méot, A. (2011). Russian norms for name agreement, image agreement for the colorized version of the Snodgrass and Vanderwart pictures and age of acquisition, conceptual familiarity, and imageability scores for modal object names. Behavior Research Methods, 43, 1085–1099. doi:10.3758/s13428011-0121-9 Vitkovitch, M., & Tyrrell, L. (1995). Sources of disagreement in object naming. Quarterly Journal of Experimental Psychology, 48A, 822– 848. doi:10.1080/14640749508401419 Weekes, B. S., Shu, H., Hao, M., Liu, Y., & Tan, L. H. (2007). Predictors of timed picture naming in Chinese. Behavior Research Methods, 39, 335–342. doi:10.3758/BF03193165 Zevin, J. D., & Seidenberg, M. S. (2002). Age of acquisition effects in word reading and other tasks. Journal of Memory and Language, 47, 1–29. doi:10.1006/jmla.2001.2834