Development and Validation of the Voice Fatigue Handicap Questionnaire (VFHQ): Clinical, Psychometric, and Psychosocial Facets *,†Nico Paolo Paolillo and ‡Giuseppe Pantaleo, *Lecco and yzMilan, Italy Summary: Objectives/Hypothesis. Development and validation of the vocal fatigue handicap questionnaire (VFHQ), a self-administered brief instrument for the assessment of functional, emotional, and physical vocal fatigue (VF). Study Design. ‘‘Outcomes research’’ with item generation, item reduction, and psychometric evaluation of instrument’s key properties involving different subsamples of patients in content generation and item-selection pilot testing, followed by a main validation study. Methods. Questionnaire content generation and item reduction by selection of 30 items from an initial pool of 60 during pilot testing, followed by psychometric assessment of VFHQ’s internal consistency, test-retest reliability, construct validity, criterion validity, and clinical efficacy in a main validation study with 87 patients with voice disorders. Internal consistency was evaluated by means of Cronbach-alpha coefficients; construct and criterion validity through multiple regression analyses; test-retest reliability by administering 35 questionnaires to pathologic respondents after a 15 days’ time-interval from first questionnaire administration; clinical efficacy, (ie, instrument’s usefulness and responsiveness) by administering 20 questionnaires to patients with voice disorders before and after clinical intervention. Results. Statistical analyses established VFHQ’s high internal consistency, reliability, construct, and criterion validity, as well as efficacy, and usefulness in therapeutic practice for the assessment of clinical and psychosocial consequences of VF (spoken voice). Conclusions. VFHQ is a useful tool for the assessment of VF, which is able to quantify its functional, emotional, and physical components. Key Words: Vocal fatigue/handicap assessment–Quality of life–Effort–Social consequences of VF.

BACKGROUND AND SIGNIFICANCE The most frequent signs and symptoms of dysphonia and voice complaints are typically grouped to form a pragmatic, empirical definition of vocal fatigue (VF). The term ‘‘vocal fatigue’’ is generally associated with symptoms of vocal tiredness after voice overuse, misuse, or abuse, but there is no clear consensus in literature on the theoretical and operational meaning of VF.1,2 Further symptoms of VF also include increased effort in voicing, vocal harshness, strained voice quality, reduced pitch range, reduced vocal control, reduced vocal projection and power, voice loss, laryngeal discomfort, throat dryness, throat and neck pain, muscle tension in neck and shoulder, worsening of symptoms during the voicing day.2–6 This emphasis on specific, yet too narrowly defined aspects of VF has delayed theoretical and empirical integration in terms of a consensual physiological and clinical conceptualization of the construct. Undeniably, an association between VF and several laryngeal pathologies has been proposed, but it is still unclear ‘‘.whether VF primarily contributes to, results from, or exists independently of other voice conditions.’’ (p. 22).2 Accepted for publication May 7, 2014. From the *Department of Otolaryngology, Head and Neck Surgery, San Leopoldo Mandic Hospital of Merate, Lecco, Italy; yUniversit
a degli Studi di Milano, Milan, Italy; and the zFaculty of Psychology, Vita-Salute San Raffaele University of Milan, Milan, Italy. Address correspondence and reprint requests to Giuseppe Pantaleo, San Raffaele University of Milan, Faculty of Psychology, I-20132, Milan, Italy. E-mail: pantaleo. [email protected] Journal of Voice, Vol. 29, No. 1, pp. 91-100 0892-1997/$36.00 Ó 2015 The Voice Foundation http://dx.doi.org/10.1016/j.jvoice.2014.05.010

Objective findings obtained through videolaryngostroboscopic examinations in patients with VF revealed the presence of anterior glottal chinks,2,7 abnormal spindle-shaped closure,7 increased bilateral vocal fold edema following a loud reading task.8,9 The lateral cricoarytenoid (LCA) muscles activity reduces during VF and, simultaneously, the activity of thyroarytenoid (TA) and cricothyroid muscles increases to compensate for the decrease in activity in LCA muscles. The increase in TA muscles activity tenses and stretches the vocal folds to stabilize the adduction force,10 inducing an overwork of these muscles and thus facilitating VF onset. According to Titze11–13 there should be several physiological and biomechanical causes underlying VF, which would lead to a simple partition of VF into laryngeal muscle fatigue (neuromuscular fatigue), and laryngeal tissue fatigue (increased vocal fold viscosity, reduced blood circulation, nonmuscular tissue strain). Still another aspect of VF should be represented by respiratory muscle fatigue. A really intriguing idea in this respect is represented by the possibility of a genetic disposition for VF, called ‘‘tender larynx,’’14 although as yet this condition is difficult to identify because the markers for its recognition are still unclear. The vocal healing for fatigue recovery rests on a repairing mechanism which is comparable to a chronic dermal wound repair. The daily (mis)use of voice provokes a reiterated damage that maintains active the healing mechanism for a constant repair, even in case of no pathologies. Thus, intense vocal load results in augmented VF and, in turn, in specific damages (e.g., postvocal load edema), which would then considerably lengthen the customary healing time.15 Although the exact

92

Journal of Voice, Vol. 29, No. 1, 2015

process of VF generation is still debated among scholars, it seems fair to assume, with different authors, that prolonged voice use, phonation at high sound pressure level and high pitch, would all induce repeated microtrauma and, in turn, the subjective experience of VF.16–19 Consequently, given a specific predisposition to glottic damages deriving from low efficiency of the previously mentioned repair mechanisms—such as in the case of patients with ‘‘tender larynx’’—the recovery time from a fatiguing episode involving vocal load should be much longer than in the case of patients with ‘‘normal larynx.’’ Ideally, an early identification of VF, and of its predisposing conditions, would thus be paramount to facilitate prevention from those chronic conditions that could cause and maintain voice disorders, particularly for those occupational voice users who are especially inclined to the risk of chronic VF.4 Importantly, VF also affects, in many everyday situations, emotional, psychosocial, and working conditions. Up to now, no specifically-created instruments exist to quantify and selfassess emotional, psychosocial and working consequences of voice fatigue. As Table 1 shows, a significant amount of spoken voice-related quality of life tests, such as the voice handicap index,20 voice rating scale,21 vocal effort scale,21 voice symptom scale,22 vocal performance questionnaire,23,24 contain items that include only indirectly words or questions regarding a thorough assessment of spoken voice fatigue. In such statements we can find words as ‘‘vocal effort,’’ ‘‘strain,’’ ‘‘tiredness,’’ ‘‘prolonged talking,’’ and so on—that is, words that relate to what usually respondents ‘‘feel.’’ However, this could not be enough to assess and quantify spoken voice fatigue. It seems also essential to know ‘‘when’’ and ‘‘where’’ symptoms occur, what subjects ‘‘do’’ with voice because of VF, what they do to ‘‘avoid’’ fatigue, ‘‘how’’ VF influences emotions, psychological functioning, personal, and social life, in various private and social circumstances. All these features of VF seems critical in clinical and rehabilitative practice, not only to assess and quantify VF and its distinctive manifestations, but also to assure an adequate follow-up during patients’ therapy and rehabilitation. Most needed is thus a psychometrically validated self-report instrument, which would enable researchers and clinicians to

assess pivotal and distinctive aspects of spoken voice fatigue such as, specifically, its ‘‘functional,’’ ‘‘emotional,’’ and ‘‘physical’’ components. Such an instrument would not only serve these scopes, but would also enable clinicians to identify persons with comparatively high(er) scores on VFHQ, ie, persons potentially at risk of VF and subsequent vocal folds damage, deserving closer clinical inspection and evaluation. The aim of the reported study was therefore to construct and validate a ‘‘vocal fatigue handicap questionnaire’’ (VFHQ), based on strict convergence of distinct conceptual and psychometric criteria, with the explicit goal of proving the instrument’s high degree of (1) internal ‘‘consistency,’’ (2) test-retest ‘‘reliability,’’ (3) construct and criterion ‘‘validity,’’ and (4) degree of clinical ‘‘efficacy,’’ or ‘‘practical relevance’’ in patients’ selfassessment of symptoms and related psychosocial consequences. To this end, we followed standard three-stage procedures of item generation, item reduction, and psychometric evaluation.25,26 MATERIALS AND METHODS Item generation and item reduction through pilot testing All procedures used in item generation, item reduction, pilot testing, and in the main validation study were approved by the medical direction of San Leopoldo Mandic Hospital in Merate-Lecco, Italy. All participants signed the Informed Consent Form. Each procedure was run in accordance to the guidelines of the Scientific Advisory Committee of the Medical Outcomes Trust.23,24 Preliminary procedures in VFHQ development consisted in the generation of an initial pool of 60 items, according to three broadly accepted—and clinically relevant—content domains in the assessment of spoken voice pertaining, respectively, to ‘‘functional’’ (20 items), ‘‘emotional’’ (20 items), and ‘‘physical’’ (20 items) aspects of voice use. Approximately two-thirds of the items were generated by three experienced laryngologists and a speech-language pathologist. These items were created on the basis of content analyses of clinical interviews conducted with a preliminary sample of 20 patients (mean age ¼ 44.28; SD ¼ 15.04; eight males and 12 females; pathology type: two

TABLE 1. Items Typically Used to Identify Vocal Fatigue (VF) in Common Voice-Related Quality of Life Tests Voice Handicap Index (Jacobson et al 1997)20 Voice Rating Scale (Wingate et al 2007)21

Vocal Effort Scale (Wingate et al 2007)21 Voice Symptom Scale (Deary et al 2003)22 Vocal Performance Questionnaire (Deary et al 2004; Paulinelli et al 2012)23,24

(P14.) ‘‘I feel as though I have to strain to produce voice’’ (P20.) ‘‘I use a great deal of effort to speak’’ Degree to which my voice tires after prolonged talking Degree to which my voice quality changes after prolonged talking Degree to which I lose my voice after prolonged talking Degree to which my throat feels sore after prolonged talking The degree of effort I have needed to speak at work this week has been The degree of effort I have needed to speak socially this week has been (I22.) ‘‘Do you find effort of speaking tiring?’’ (I35.) ‘‘Do you feel to have to strain to produce your voice?’’ (I4.) ‘‘Do you find it an effort to talk?’’

Notes: Italics added. The grouping and position of original items within each scale are reported in parentheses.

Nico Paolo Paolillo and Giuseppe Pantaleo

Development and Validation of the VFHQ

patients (10%) with neurogenic dysphonia, nine (45%) with structural dysphonia, four (20%) with inflammatory dysphonia, and five (25%) with functional dysphonia). Each interview was conducted with the precise intent of discovering patients’ most frequent remarks on spoken voice fatigue. The remaining items were instead directly obtained, always by the same group of experts, from inspection of the scientific literature on voice disorders in dysphonic patients. From a content-related, theoretical point of view, the items of the ‘‘functional’’ subscale were generated to learn ‘‘when’’ patients feel VF; the place ‘‘where’’ they feel VF; what patients ‘‘do with voice’’ because of VF; and what they do to ‘‘avoid’’ fatigue. These items describe the influence of VF in daily life and activities such as, for instance, at work or during leisure time. The functional subscale evaluates the beginning and the duration of symptoms, and the extent to which patients modify their vocal habits to avoid or to delay the onset of VF. Complementing this information, the questions composing the ‘‘emotional’’ subscale were instead created to mark the influence of VF on patients’ personal and social life in various environmental and temporal situations. The items of this subscale were thus created to signal potential fears, anxieties, and personal discomfort associated with VF in a wide collection of daily individual and broader social contexts and situations. Finally, the items of the ‘‘physical’’ subscale were generated to inquire into ‘‘how’’ VF affects vocal quality and influences, thereby, vocal efficiency. Through detailed content analysis of focused clinical interviews, and careful inspection of related scientific literature (as previously mentioned), a total of 60 items was obtained. These items were assembled to form a preliminary version of the VFHQ. The format of each item was the same as that reported in Table 2. After questions generation, item reduction took place through further extensive pilot testing. To this end, the initial pool of 60 items was administered to 30 patients (mean age ¼ 43.82; SD ¼ 12.40; six males and 14 females; pathology type: two patients (6.67%) with neurogenic dysphonia, 15 (50%) with structural dysphonia, seven (23.33%) with inflammatory dysphonia, and six (20%) with functional dysphonia). Exclusion criteria were represented by the presence of any neurological, psychological, and/or cognitive disorders that would not enable the proper administration of the questionnaire, or would in any form interfere with the comprehension of instructions). Patients were instructed to answer all 60 questions, and to circle items that in their opinion better described the effects of spoken voice fatigue on their voices and social life. A reduced pool of 30 items was thus obtained through selection (1) on the basis of the ‘‘Cronbach’s-alpha-if-item-deleted’’ criteria and procedure, and (2) on the basis of items’ face validity. The remaining 30 items were then grouped according to the aforementioned three established theoretically and clinically relevant content domains (functional, emotional, and physical aspects of VF), with the explicit intent of validating this a priori partition in a formal psychometric main validation study. Before using the 30-items VFHQ questionnaire in the main validation study, however, a sample of 20 additional patients (mean age ¼ 44.32; SD ¼ 15.64; 12 males and 18 females;

93

pathology type: three patients (15%) with neurogenic dysphonia, 10 (50%) with structural dysphonia, three (15%) with inflammatory dysphonia, and four (20%) with functional dysphonia) was asked to answer all 30 questions to ensure that each item was still fully comprehensible in the reduced version of the questionnaire (i.e., in the new interpretative and judgmental context), and that patients could complete the questionnaire alone, without the need of any special dedicated or technical assistance. As a further measure of caution in VFHQ’s generation and development, patients were also instructed to mark and delete, with a pen, all sentences they could not—for any reason—understand or answer. No sentences were deleted, or otherwise indicated as difficult/impossible to understand or answer. By adopting this complete body of procedural/ developmental measures, we fulfilled the preliminary two stages of items ‘‘generation’’ and item ‘‘reduction’’ required for questionnaires’ content development, as suggested by the official guidelines for ‘‘patient-reported outcomes measures’’ (PROMs), recently instituted (1) to reduce ambiguities in the wording of psychometric instruments, (2) to confirm instruments’ appropriateness, and (3) to determine acceptability and mean time required for instrument completion (5–10 minutes, in the case of the reduced, 30-items version of VFHQ).26 Main study: participants and procedure A total sample of 87 patients (mean age ¼ 45.33; SD ¼ 16.83; 24 males and 63 females) with voice disorders—six patients (6.90%) with neurogenic dysphonia, 44 (50.57%) with structural dysphonia, 19 (21.84%) with inflammatory dysphonia, and 18 (20.69%) with functional dysphonia—, filled-in the reduced version of the VFHQ questionnaire (30 items—Table 2). The total population included and balanced both ‘‘workers’’ (57.50%), and ‘‘nonworkers’’ (42.50%). Further, we took special care in administering the VFHQ to a comprehensive array of elite vocal performers (e.g., artists), professional voice users (e.g., clergy, teachers, lecturers), nonvocal professionals (e.g., lawyers, physicians, businessmen), and nonprofessionals (retired, unemployed, students, housewives). Further, a subsample of 37 patients took the test twice (pretest vs retest), in a time-span of a minimum of 13 to a maximum of 37 days from the first administration. Of these 37 patients, 17 underwent clinical treatment (medical therapy, surgery, or speech therapy for at least eight sessions, on a weekly basis) between the first and the second administration of the questionnaire, whereas the remaining 20 patients served as the nontreated comparison group. All patients underwent phoniatric examination. On questionnaire completion, patients also rated the severity of their symptoms of VF on a separate 4-point Likert-type scale (0 ¼ ‘‘normal voice’’; 1 ¼ ‘‘mildly impaired voice’’; 2 ¼ ‘‘moderately impaired voice’’; 3 ¼ ‘‘severely impaired voice’’). In strict analogy with the procedures adopted in the development and validation of the ‘‘voice handicap index’’ (VHI),20,26 a well-known and widely used psychometric instrument, the selected VFHQ items were grouped a priori into three content domains, representing ‘‘functional’’ (10 items), ‘‘emotional’’ (10 items), and ‘‘physical’’ (10 items) facets of voice fatigue, respectively. In support of this partition, construct validity

94

Journal of Voice, Vol. 29, No. 1, 2015

TABLE 2. The Voice Fatigue Handicap Questionnaire (VFHQ)
che e
indicato nelle seguenti dichiarazioni Cerchia la risposta che indica quanto frequentemente si verifica cio Circle the answer that indicates how frequently you experience what indicated in the statements below (0 ¼ Mai/Never 1 ¼ Quasi Mai/Almost Never 2 ¼ Qualche Volta/Sometimes 3 ¼ Quasi Sempre/Almost Always 4 ¼ Sempre/ Always) Functional Subscale
professionale/Effects of VF on the job Impatto della fatica vocale sull’attivita F1 La fatica vocale si manifesta fin dall’inizio della giornata 0 1 2 3 4 My vocal fatigue appears from the beginning of the day F2 La fatica vocale subentra nel corso della giornata 0 1 2 3 4 My vocal fatigue appears as the day goes by
manifesta al termine della giornata F3 La fatica vocale e 0 1 2 3 4 My vocal fatigue appears at the end of the day F4 La fatica vocale si manifesta prevalentemente in ambito lavorativo 0 1 2 3 4 My vocal fatigue appears mostly at work F5 La fatica vocale si manifesta prevalentemente in ambito extralavorativo 0 1 2 3 4 My vocal fatigue appears mostly when I am not working F6 Sono costretto a sforzarmi nel produrre la voce 0 1 2 3 4 I have to strain to produce voice F7 Tossisco/schiarisco la voce durante la giornata 0 1 2 3 4 I cough/throat clear during the day F8 Modifico le mie abitudini vocali e comportamentali pur di ridurre la stanchezza vocale 0 1 2 3 4 I change my vocal and behavioral habits to avoid getting my voice tired F9 Limito l’utilizzo della voce in ambito sociale 0 1 2 3 4 I speak less in social situations F10 Per mascherare l’affaticamento vocale sono costretto a terapie 0 1 2 3 4 I constantly have to take medication to deal with my vocal fatigue Emotional Subscale Impatto psicologico/Psychological impact
nel parlare E1 Il timore di affaticare la voce limita la mia liberta 0 1 2 3 4 I speak less because I am afraid of wearing out my voice
vocali E2 Ho la costante impressione di non essere nel pieno possesso delle mie capacita 0 1 2 3 4 I constantly feel that I am not at my full voice capacity E3 Temo che il rendimento vocale si deteriori nel corso della giornata 0 1 2 3 4 I am afraid that my voice will get worse as the day goes by E4 Il timore del deterioramento vocale acuisce la mia ansia da prestazione in ambito lavorativo 0 1 2 3 4 The fear of damaging my voice makes me anxious at work E5 La tendenza ad affaticarmi mi mette a disagio in ambito lavorativo 0 1 2 3 4 My vocal fatigue makes me feel handicapped at work E6 La tendenza ad affaticarmi mi mette a disagio in ambito extralavorativo 0 1 2 3 4 My vocal fatigue makes me feel handicapped in my social and personal life E7 La tendenza ad affaticarmi mi mette a disagio in ambito familiare 0 1 2 3 4 My vocal fatigue makes me feel handicapped at home E8 Il timore di affaticare la voce incide sul mio stato emotivo 0 1 2 3 4 The fear of wearing out my voice affects my emotions E9 Il timore del deterioramento vocale incide sul mio stile di vita 0 1 2 3 4 The fear of damaging my voice affects my life style E10 Le persone che mi circondano non comprendono il mio stato di disagio 0 1 2 3 4 People do not understand my handicap Physical Subscale
vocale/Perception of quality of voice Percezione della qualita P1 La voce si interrompe/rompe improvvisamente durante l’utilizzo 0 1 2 3 4 My voice breaks while speaking
nel controllare la voce P2 Ho difficolta 0 1 2 3 4 I have difficulty controlling my voice P3 La voce parlata diventa flebile 0 1 2 3 4 My speaking voice becomes weak (Continued )

Nico Paolo Paolillo and Giuseppe Pantaleo

95

Development and Validation of the VFHQ

TABLE 2. (Continued ) P4 P5 P6 P7 P8 P9 P10

La voce parlata diventa soffiata My speaking voice becomes breathy La voce parlata diventa rauca e/o sporca My speaking voice becomes hoarse La voce parlata diventa spinta/sforzata My speaking voice becomes strained La voce parlata diventa stimbrata, meno ricca My speaking voice becomes dull/less vibrant Avverto sensazioni di costrizione durante l’utilizzo della voce I feel tightness in my throat when I use my voice Avverto fastidio e/o tensione e/o dolore a livello di collo e/o di spalle I feel discomfort/tension/pain in my neck and/or shoulders La resa vocale varia nell’arco della giornata My voice varies throughout the day

was then assessed through multiple regression analyses (see ‘‘construct validity’’ section in the following). Internal consistency The internal consistency of the VFHQ was assessed using Cronbach a coefficient. For this analysis the VFHQ scores of the total sample of 87 patients were used. Test-retest reliability A Pearson product-moment correlation coefficient was computed to determine the test-retest reliability (i.e., the stability over time in absence of induced or otherwise intervened variations) of the VFHQ subscales and total score. For this analysis only the VFHQ scores of 20 patients who took the test twice, in a time-span of approximately 15 days (mean ¼ 15.00; SD ¼ 1.21) from the first administration—but who were of course ‘‘not treated’’ in between—were used. Construct validity Construct validity of VFHQ was determined by three separate multiple regression analyses. In each analysis the ‘‘criterion’’ variable was represented by the retest scores of either (1) the functional, (2) the emotional, or (3) the physical dimension, whereas the ‘‘predictors’’ were represented by patients’ pretest scores on each of the three VFHQ subdimensions, which were entered simultaneously in the regression equation. If the a priori grouping of VFHQ items into three separate factors is conceptually valid—i.e., if VFHQ has construct validity— then patients’ retest scores should be uniquely predicted only by the corresponding pretest predictive dimension—but not by the other two remaining dimensions. For instance, patients’ VFHQ scores on the ‘‘functional’’ dimension at retest should be uniquely predicted only by their scores on the ‘‘functional’’ dimension at pretest, but not by their pretest scores on the ‘‘emotional,’’ or ‘‘physical’’ dimensions. The same holds true for the other two proposed dimensions. For this analysis only the scores of 20 patients who took the test twice in a timespan of approximately 15 days (mean ¼ 15.00; SD ¼ 1.21)

0

1

2

3

4

0

1

2

3

4

0

1

2

3

4

0

1

2

3

4

0

1

2

3

4

0

1

2

3

4

0

1

2

3

4

from the first administration, without any treatment in between, were considered. Criterion validity The capacity of VFHQ total score—and of its single components—to predict patients’ self-rated severity of symptoms of VF was assessed through simple and multiple regression analyses. Patients’ self-rated severity of symptoms represented the criterion variable against which VFHQ predictive power had to be demonstrated to establish criterion validity. Thus, in the overall regression analysis, variations in patients’ total VFHQ scores were used to predict variations in selfrated severity scores. Further, in subsequent analyses, patients’ scores on each of the three separate components of VF (functional, emotional, and physical) were simultaneously entered in a multiple regression equation, to prove the distinct and specific predictive power of each component with respect to patients’ self-rated severity of symptoms. For this analysis only the VFHQ scores of the 70 nontreated patients were used. Clinical efficacy and practical relevance To assess the clinical efficacy, and thus the usefulness and practical relevance of VFHQ, we monitored variations in patients’ total VFHQ scores ‘‘before’’ and ‘‘after’’ clinical intervention, and compared them with the expected absence of variations in a comparable group of patients who did not undergo any intervention between pretest and retest administration of VFHQ. Thus, we selected two groups of patients for this analysis, one formed by 17 patients that underwent clinical treatment and the other by 20 nontreated patients. Both groups of patients answered the VFHQ questions twice, at ‘‘pretest’’ and at ‘‘retest.’’ Retest took place after a minimum of 13 to a maximum of 37 days from the first administration. RESULTS Patients’ answers to the VFHQ were summed and then averaged into three separate scale components (‘‘functional,’’ ‘‘emotional,’’ and ‘‘physical’’), to reflect the three hypothesized facets of VF. Also a total VFHQ average score was obtained by

96 summing and then averaging patients’ answers to all questionnaire items (30 items). All scale and subscale scores are thus directly comparable, because they are expressed as average scores (all ranging 0–4). If necessary, these scores can be easily converted into subscale sum scores (ranging 0–40) by multiplying them by 10, and into total VFHQ sum scores (ranging 0–120) by multiplying them by 30. In validating the VFHQ, however, we will make strict use of average scores (all ranging 0–4), because they retain the original metric of each VFHQ single item, at any level of measurement (single items, separate subscales, and total VFHQ average scores), thus allowing steady and direct comparability of results when evaluating the results of either single items, separate subscales, or total VFHQ average scores. VFHQ internal consistency analysis The Cronbach a coefficient value computed for the total VFHQ score was .95; for the functional, emotional and physical subscales, the Cronbach a coefficients values were a ¼ .86, a ¼ .92 and a ¼ .86, respectively. It might be noted that, in the literature, a value of 0.80 is considered ‘‘good’’ and greater than 0.90 ‘‘excellent,’’ whereas a value around 0.70 is often considered ‘‘satisfactory.’’26 VFHQ test-retest reliability analysis Test-retest reliability was found to be strong for the functional (r ¼ 0.97), emotional (r ¼ 0.84), and physical (r ¼ 0.92) test subscales, respectively (all ps < 0.001). Test-retest reliability for the total VFHQ score was also very high (r ¼ 0.94, p < 0.001). Further, we carefully controlled for any possible gender difference on subscales and total VFHQ average scores at pretest and retest, and found no significant discrepancies between men and women in this respect (all ts < 1.01, and all ps > 0.32). In the absence of such differences, results of testretest reliability analyses performed both on total VFHQ average scores and on subscales scores (‘‘functional,’’ ‘‘emotional,’’ ‘‘physical’’) must be considered equivalent and stable over time both for males and females. VFHQ construct validity analysis Construct validity was assessed in three separate multiple regression analyses. As expected, in the first multiple regression equation [model fit: R ¼ 0.96; R2 ¼ 0.95; Adj. R2 ¼ 0.94; Se ¼ 0.18; F(3, 19) ¼ 103.45, p < 0.001], patients’ functional scores at retest were uniquely predicted by their functional scores at pretest (b ¼ .90, t ¼ 9.27, p < 0.001), but not by their emotional (b ¼ .05, t ¼ 0.60, non-significant [n.s.]), nor by their physical (b ¼ .14, t ¼ 1.60, n.s.) scores at pretest. Analogously, in the second multiple regression equation [model fit: R ¼ 0.85; R2 ¼ 0.72; Adj. R2 ¼ 0.67; Se ¼ 0.43; F(3, 19) ¼ 13.72, p < 0.001], patients’ emotional scores at retest were uniquely predicted by their emotional scores at pretest (b ¼ .70, t ¼ 3.42, p < 0.005), but not by their functional (b ¼ .07, t ¼ 0.07, n.s.), nor physical (b ¼ .22, t ¼ 1.00, n.s.) scores at pretest. Finally, in the third multiple regression equation [model fit: R ¼ 0.94; R2 ¼ 0.89; Adj. R2 ¼ 0.86; Se ¼ 0.27; F(3, 19) ¼ 41.18, p < 0.001], patients’ physical

Journal of Voice, Vol. 29, No. 1, 2015

scores at retest were predicted by their physical scores (b ¼ 1.01, t ¼ 7.28, p < 0.001) and, to a lesser extent, also by their emotional scores at pretest (b ¼ .31, t ¼ 2.37, p < 0.04), but not by their functional (b ¼ .16, t ¼ 1.07, n.s.) scores at pretest. VFHQ criterion validity analysis The predictive power of the total VFHQ scores, e.g., its capacity to predict self-rated symptoms of VF (the ‘‘criterion’’), was found to be very high (b ¼ .83, t ¼ 12.30, p < 0.001), as demonstrated by a simple regression analysis [model fit: R ¼ 0.83; R2 ¼ 0.69; Adj. R2 ¼ 0.68; Se ¼ 0.40; F(1, 69) ¼ 151.24, p < 0.001]. Further, a multiple regression analysis [model fit: R ¼ 0.84; R2 ¼ 0.71; Adj. R2 ¼ 0.70; Se ¼ 0.39; F(3, 69) ¼ 53.31, p < 0.001] revealed that each single VFHQ component uniquely contributed to the prediction of the severity of self-rated VF. The stronger predictor of the criterion variable, however, was the ‘‘functional’’ component (b ¼ .52, t ¼ 4.64, p < 0.001), followed by the ‘‘emotional’’ (b ¼ .21, t ¼ 2.46, p < 0.02) and ‘‘physical’’ (b ¼ .21, t ¼ 1.99, p ¼ 0.05) components, respectively. VFHQ clinical efficacy and practical relevance analysis Preliminary sample analyses showed that gender was equally distributed within subsamples of treated versus nontreated patients (six males and 14 females within the subsample of nontreated patients vs six males and 11 females within the subsample of treated patients; c2 ¼ 0.12, n.s.). The two groups of patients were also comparable with respect to age (mean age of nontreated patients ¼ 40.85, SD ¼ 15.04; mean age of treated patients ¼ 42.12, SD ¼ 17.69, t(35) ¼ 0.24, p ¼ 0.82, n.s.), and pathology type: one patient (5.00%) with neurogenic dysphonia, 11 patients (55.00%) with structural dysphonia, two (10.00%) with inflammatory dysphonia, and six (30.00%) with functional dysphonia within the nontreated group versus two patients (11.76%) with neurogenic dysphonia, nine (52.94%) with structural dysphonia, five (29.41%) with inflammatory dysphonia, and one patient (5.88%) with functional dysphonia within the treated group (Fisher exact test, p ¼ 0.14, n.s.). Finally, the breakdown by therapy type within the subsample of treated patients yielded the following results: seven (41.18%) patients underwent surgery, five (29.41%) speech therapy, and the remaining five patients (29.41%) were treated with drugs. VFHQ clinical efficacy, and thereby its practical relevance, was thus tested through the nonparametric Wilcoxon signedrank test for related samples. The statistical analysis revealed that, as expected, there was no variation in total VFHQ scores between the first (mean total VFHQ score ¼ 1.44; SD ¼ 0.66) and the second (mean total VFHQ score ¼ 1.45; SD ¼ 0.65) questionnaire administration for patients who did not undergo clinical treatment (Z ¼ 0.68, n.s.). However, the anticipated significant difference clearly emerged for treated patients, for their total VFHQ score decreased significantly from pretest (mean total VFHQ score ¼ 2.10; SD ¼ 0.72) to retest (mean total VFHQ score ¼ 0.87;

Nico Paolo Paolillo and Giuseppe Pantaleo

Development and Validation of the VFHQ

SD ¼ 0.76), Z ¼ 3.62, p < 0.001). Further, a nonparametric focused Mann-Whitney test for independent samples, ie, a statistical test we used to compare total VFHQ scores of treated versus nontreated patients at the retest, confirmed that treated patients reported significantly lower VFHQ total scores than their untreated counterparts. (Z ¼ 2.46, p < 0.02). The time-interval between test and retest was higher for treated (mean ¼ 31.12 days; SD ¼ 4.11) than for nontreated patients (mean ¼ 15.00; SD ¼ 1.21), t(18.38) ¼ 15.61, P < 0.001). We thus controlled statistically for possible unwanted effects of differing time-intervals on VFHQ scores, by implementing a covariance analysis (ANCOVA) in a two by two repeated-measures design, whereby the first (‘‘between-subjects’’) factor was represented by the partition between treated versus nontreated patients, the second (‘‘withinsubjects’’) factor by patients’ VFHQ average total scores at pretest versus retest, and the covariate by the time-interval, expressed in days, between pretest and retest. As can be seen from Figure 1, through the statistical pairing of treated versus nontreated patients with respect to time-intervals, we replicated the general effect of treatment efficacy on patients’ VFHQ scores, as shown by the presence of the significant statistical interaction between the two factors, ‘‘group of patients’’ and ‘‘pretest versus retest,’’ while controlling—this time—for the possible confounding influence of the covariate (ie, the differing time-intervals between the two groups of patients), F(1, 34) ¼ 17.29, p < 0.001. Whereas patients’ average VFHQ total scores remained unchanged between pretest (estimated marginal mean ¼ 0.84, Se ¼ 0.32) and retest (estimated marginal mean ¼ 1.03, Se ¼ 0.33) for nontreated patients, F < 1, the scores of treated patients clearly dropped from pretest (estimated marginal mean ¼ 2.81, Se ¼ 0.37) to retest (esti-

4.00

Total VFHQ Average Scores

3.00

Treated Patients

2.00

1.00 Untreated Patients 0.00

Pretest

Retest

FIGURE 1. Treated versus nontreated patients total VFHQ average scores (ranging 0–4) considered at pretest versus retest. The significant drop observed in the scores of treated patients after treatment (retest) proves the clinical efficacy of the VFHQ in detecting the anticipated change; test-retest reliability is confirmed, instead, by the stability of the scores of nontreated patients considered at pretest versus retest.

97

mated marginal mean ¼ 1.36, Se ¼ 0.39), exactly in the expected direction, F(1, 15) ¼ 3.82, p ¼ 0.035. Table 3 shows that this pattern of results was clearly replicated for each of the three subscales composing the VFHQ (interaction effect for the ‘‘functional’’ subscale: F(1, 34) ¼ 14.37, p < 0.001; interaction effect for the ‘‘emotional’’ subscale: F(1, 34) ¼ 15.79, p < 0.001; interaction effect for the ‘‘physical’’ subscale: F(1, 34) ¼ 7.22, p < 0.012). Thus, taken as a whole, the previously mentioned total configuration of results further testify, from a different analytic angle, to the clinical efficacy and practical relevance of the questionnaire.

DISCUSSION The aim of this study was to develop and validate the ‘‘Vocal Fatigue Handicap Questionnaire’’ (VFHQ), a self-administered, easy-to-use instrument for the assessment of ‘‘functional,’’ ‘‘emotional,’’ and ‘‘physical’’ spoken voice fatigue. To this end we run extensive pilot testing for items generation and selection, and then demonstrated the instrument’s intrinsic psychometric properties in a systematic main validation study. Specifically, we proved VFHQ’s high degree of internal consistency, testretest reliability, construct and criterion validity, and its clinical efficacy and, thus, practical relevance. The internal consistency of VFHQ appeared high, with Cronbach a coefficient values ranging from .86, for the ‘‘functional’’ and ‘‘physical’’ subscales, to .91, for the ‘‘emotional’’ subscale. The overall Conbach a coefficient value was extremely high. Thus, the internal consistency of the VFHQ as a measuring instrument—ie, as a ‘‘patient-reported outcomes measure’’ (PROMs),27— and of its separate components, is assured. Test-retest reliability of VFHQ was guaranteed too, for Pearson product-moment correlation coefficients ranged 0.84–0.97 with respect to each of the three separate subscales, and the total reliability score showed an extremely high reproducibility value of 0.94. Further, the demonstrated predictive power of each test component (‘‘functional,’’ ‘‘emotional,’’ and ‘‘physical’’) emerged in the course of the ‘‘construct’’ validity analyses, also proves—from a different analytic angle—the fundamental test-retest reliability of each of the three components of VFHQ. An additional demonstration of VFHQ’s test-retest reliability was also apparent during the assessment of the instrument’s clinical efficacy and practical relevance, in which the total VFHQ scores obtained during the first (pretest) and the second (retest) administration remained stable for patients who did not undergo any clinical treatment (whereas they did vary, as one would expect, for treated patients). Thus, both the evidence coming from the assessment of the magnitude and significance of Pearson coefficients, and the evidence coming from related statistical analyses intended to prove the other distinctive features of VFHQ, confirm that VFHQ is a highly reliable instrument, whose scores can be confidently reproduced over time. Construct validity of VFHQ was apparent in the results of multiple regression analyses, in which patients’ scores on each of the three VFHQ subdimensions were uniquely predicted by the corresponding scores on the pretest

98

Journal of Voice, Vol. 29, No. 1, 2015

TABLE 3. Results of control ANCOVAs, Leading to Estimated Marginal Means (EMM) and Standard Errors (Se) for Treated and Nontreated Patients Considered at Pretest and Retest

Treated patients Nontreated patients

Total VFHQ Average Scores

Functional Subscale

Emotional Subscale

Pretest

Pretest

Pretest

Retest

Retest

Retest

Physical Subscale Pretest

Retest

EMM

Se

EMM

Se

EMM

Se

EMM

Se

EMM

Se

EMM

Se

EMM

Se

EMM

Se

2.81 0.84

0.37 0.32

1.36 1.03

0.39 0.33

3.24 0.92

0.40 0.35

1.53 1.22

0.40 0.35

2.41 0.26

0.42 0.36

1.00 0.59

0.43 0.37

2.76 1.34

0.40 0.35

1.56 1.30

0.42 0.36

Notes: After controlling for possible time-influences, scores of treated patients were consistently lower at retest than at pretest, whereas scores of nontreated patients remained substantially unchanged, this confirming (a) the clinical efficacy of the VFHQ in detecting predicted change, and (b) test-retest reliability of the instrument where change was not expected to occur. All scale and subscales scores range 0–4.

subdimensions. This was generally true, with an interesting minor exception for the physical subdimension. Patients’ functional scores at retest were uniquely predicted by their pretest scores on the corresponding functional test component, but not by their scores on the remaining two dimensions (emotional, physical). Similarly, patients’ emotional scores at retest were uniquely predicted by their pretest scores on the corresponding emotional test component, but not by their scores on the remaining two dimensions (functional, physical). Interestingly, patients’ physical scores at retest were largely predicted by their pretest scores on the corresponding physical component, and—to a much lesser extent—negatively predicted by their pretest scores on the emotional test component (whereas functional scores at pretest did not predict any variation in physical scores at retest). Thus, whereas the results of multiple regression analyses converge in proving the construct validity of VFHQ, the emotional component of VF also seems to play a role in predicting patients’ appraisals of some physical facets of VF. Specifically, the stronger the perceived impact of the emotional (i.e., social) component of VF, the weaker the perceived burden of physical symptoms. In the same vein, the weaker the perceived impact of the emotional/social component of VF, the stronger the perceived impact of physical symptoms—as if a mechanism of psychological balancing and compensation between the two sets of symptoms was at work in the present investigation. The criterion validity of VFHQ was demonstrated by its manifest capacity to predict the severity of patients’ self-rated symptoms of VF. This was true both when using VFHQ’s total score to predict self-rated severity of symptoms, and when using simultaneously all three separate predictive components (‘‘functional,’’ ‘‘emotional,’’ and ‘‘physical’’) to predict severity of symptoms. More specifically, the functional component proved to be a stronger predictor of self-rated severity if compared with the emotional and physical components. These two latter components, however, still revealed their unique and distinctive predictive power in contributing to the estimation of patients’ variations in self-rated severity scores—thus also in this case establishing and confirming the criterion validity of the instrument from a still different analytic angle. Following the lead of Brehm,28 and the classical contribution of

Loevinger,29 in the present study, criterion validity was established by deliberately using only one single item as the ‘‘criterion’’. Future studies may thus profit from comparing the performance of VFHQ with that of other psychometric instruments of common use in the field. Such comparisons would be especially informative, in that they would enable scholars to establish, for instance, instruments’ reciprocal convergent versus discriminant validity, and other mutual or differentiating psychometric properties, thus adding to our general understanding of the key features of each of the instruments under scrutiny. Clinical efficacy and, thus, practical relevance of VFHQ was proved by comparing the presence (vs absence) of variations in VFHQ total scores for treated (vs nontreated) patients. Whereas, as expected, nontreated patients did not show any variation in total VFHQ scores, treated patients showed significantly lower scores after treatment. Notably, at retest, VFHQ scores of treated patients also differed significantly (i.e., were consistently lower) from those of nontreated patients, as expected. Turning to more clinical considerations, a highly-efficient instrument for an in-depth and rapid assessment of spoken voice fatigue is needed for several reasons. Not only because of the obvious professional negative consequences, and related high costs, brought about by average to severe undetected and untreated cases of spoken voice fatigue, but also because it seems important to inspect, scrutinize, and document the related and ever-present social psychological consequences of VF as well. Among these consequences we find, for instance, sensible to severe global impairments in persons’ emotional and relational balance, which too often breaks and translates in a plain worsening of people’s quality of life in the course of their everyday social relations.30–33 A precise assessment of ‘‘functional’’ VF would seem important, moreover, to highlight in which part of the day, and in which environmental context, VF occurs, how behaviors and habits are influenced by it, and which strategies people follow to avoid fatigue. A detailed appraisal of all these features is of paramount importance for establishing appropriate counseling and rehabilitation strategies. The assessment of ‘‘emotional’’ and ‘‘physical’’ VF, instead, would be valuable,

Nico Paolo Paolillo and Giuseppe Pantaleo

Development and Validation of the VFHQ

in our view, to create a reference point during clinical and rehabilitative follow-up, that clinicians and therapists could profitably use for calibrating therapy. In examining the construct of ‘‘vocal fatigue’’, we used a set of univocal operational definitions based on close scrutiny of its distinctive ‘‘functional,’’ ‘‘emotional,’’ and ‘‘physical’’ facets (i.e., the ‘‘When,’’ ‘‘Where,’’ ‘‘What,’’ and ‘‘How’’ of VF). Such a characterization of the construct might represent an important starting point for stimulating further theoretical and empirical integration, also in terms of more refined converging physiological and clinical evidence. Such a collective endeavor would allow the field to come to even more stringent and increasingly less ambiguous conceptualizations of what we— scholars and practitioners—intend by ‘‘vocal fatigue’’ and its manifestations. CONCLUSIONS The reported research resulted in the creation of the ‘‘Vocal Fatigue Handicap Questionnaire’’ (VFHQ), a self-administered instrument that can be easily used and quickly scored in many clinical and research situations. VFHQ highlights and defines three specific facets of spoken voice fatigue—‘‘functional,’’ ‘‘emotional,’’ and ‘‘physical.’’ VFHQ also meets optimally all the conceptual and psychometric criteria of construct and criterion validity, internal consistency, reliability, stability and repeatability, and clinical efficacy (ie, practical usefulness and relevance). Further, the availability of such an instrument, specifically created to assess strength and severity of VF, appears particularly important today, given that research has repeatedly demonstrated that fatigue in general—thus, presumably, also ‘‘vocal’’ fatigue—plays a pivotal and complex, yet fascinating role in moderating the mobilization versus withdrawal of effort from key physiological and psychological motivational processes underlying health and personal wellbeing in the course of social interactions and social life.34–37 Prospective cohort studies using VFHQ to predict voice disorders are especially recommended. Such studies would allow, of course, a systematic screening of people who are potentially at risk of voice pathologies without current voice disorders. Furthermore, such an endeavor would also be clearly convenient to stimulate further independent assessment of VFHQ’s usefulness, capability, and potential in helping clinicians and rehabilitators to identify patients inclined to VF. If successful, such an early evidence-based indication would facilitate screening and prevention policies, especially for high-risk occupational categories. Acknowledgments Individual contribution: Both authors contributed equally to the realization of the work reported in this article (research design, data collection, data analysis, and writing of the article). REFERENCES 1. Stemple J, Stanley J, Lee L. Objective measures of voice production in normal subjects following prolonged voice use. J Voice. 1995;9:127–133. 2. Welham NV, Maclagan MA. Vocal fatigue: current knowledge and future directions. J Voice. 2003;17:21–30.

99

3. Gotaas C, Starr CD. Vocal fatigue among teachers. Folia Phoniatr (Basel). 1993;45:120–129. 4. Colton RH, Casper JK, Leonard RL. Understanding Voice Problems: A Physiological Perspective for Diagnosis and Treatment. 3rd ed. Philadelphia, PA: Lippincott Williams & Wilkins; 2006. 5. Kitch JA, Oates J. The perceptual features of vocal fatigue as self-reported by a group of actors and singers. J Voice. 1994;8:207–214. 6. Kostyk BE, Putnam Rochet A. Laryngeal airway resistance in teachers with vocal fatigue: a preliminary study. J Voice. 1998;12:287–299. 7. Eustace CS, Stemple JC, Lee L. Objective measures of voice production in 9 patients complaining of laryngeal fatigue. J Voice. 1996;10:146–154. 8. Scherer RC, Titze IR, Raphael BN, Wood RP, Ramig LA, Blager RF. Vocal fatigue in a professional voice user. In: Lawrence VL, ed. Transcripts of the Fourteenth Symposium: Care of the Professional Voice. New York, NY: The Voice Foundation; 1986:124–130. 9. Scherer RC, Titze IR, Raphael BN, Wood RP, Ramig LA, Blager RF. Vocal fatigue in a trained and an untrained voice user. In: Baer T, Sasaki C, Harris K, eds. Laryngeal Function in Phonation and Respiration. San Diego, CA: Singular Publishing Group; 1991:533–555. 10. Boucher VJ, Ayad T. Physiological attributes of vocal fatigue and their acoustic effects: a synthesis of findings for a criterion-based prevention of acquired voice disorders. J Voice. 2010;24:324–336. 11. Titze IR. Toward occupational safety criteria for vocalization. Logoped Phoniatr Vocol. 1999;24:49–54. 12. Titze IR. Vocal fatigue: some biomechanical considerations. In: Lawrence VL, ed. Transcripts of the Twelfth Symposium: Care of the Professional Voice. Part One: Scientific Papers. New York, NY: The Voice Foundation; 1984:97–104. 13. Titze IR. Principles of Voice Production. Englewood Cliffs, NJ: Prentice Hall; 1994. 14. Hunter EJ, Titze IR. Quantifying vocal fatigue recovery: dynamic vocal recovery trajectories after a vocal loading exercise. Ann Otol Rhinol Laryngol. 2009;118:449–460. 15. Robson MC, Steed DL, Franz MG. Wound healing: biologic features and approaches to maximize healing trajectories. Curr Probl Surg. 2001;38:72–140. 16. Chang A, Karnell MP. Perceived phonatory effort and phonation threshold pressure across a prolonged voice loading task: a study of vocal fatigue. J Voice. 2004;18:454–466. 17. Doellinger M, Lohscheller J, McWhorter A, Kunduk M. Variability of normal vocal fold dynamics for different vocal loading in one healthy subject investigated by phonovibrograms. J Voice. 2009;23:175–181. 18. Laukkanen AM, Ilomaki I, Leppanen K, Vilkman E. Acoustic measures and self-reports of vocal fatigue by female teachers. J Voice. 2008;22:283–289. 19. Remacle A, Finck C, Roche A, Morsomme D. Vocal impact of a prolonged reading task at two intensity levels: objective measurements and subjective self-ratings. J Voice. 2012;26:177–186. 20. Jacobson BH, Johnson A, Grywalski C, et al. The voice handicap index (VHI): development and validation. Am J Speech Lang Pathol. 1997;6:66–70. 21. Wingate JM, Brown WS, Shrivastav R, Davenport P, Sapienza CM. Treatment outcomes for professional voice users. J Voice. 2007;21:433–449. 22. Deary IJ, Wilson JA, Carding PN, MacKenzie K. VoiSS: a patient-derived Voice Symptom Scale. J Psychosom Res. 2003;54:483–489. 23. Deary IJ, Webb A, MacKenzie K, Wilson JA, Carding PN. Short, self-report voice symptom scales: psychometric characteristics of the voice handicap index-10 and the vocal performance questionnaire. Otolaryngol Head Neck Surg. 2004;131:232–235. 24. Paulinelli B, C^ortes Gama AC, Behlau M. Validation of the vocal performance questionnaire in Brazil. Rev Soc Bras Fonoaudiol. 2012;17:85–91. 25. Cano SJ, Browne JP, Lamping DL. The patient outcomes of surgery-head/ neck (POS-head/neck): a new patient-based outcome measure. J Plast Reconstr Aesthet Surg. 2006;59:65–73. 26. Schindler A, Bottero A, Capaccio P, et al. Vocal improvement after voice therapy in unilateral vocal fold paralysis. J Voice. 2008;22:113–118. 27. Branski RC, Cukier-Blaj S, Pusic A, et al. Measuring quality of life in dysphonic patients: a systematic review of content development in patientreported outcomes measures. J Voice. 2012;24:193–198. 28. Brehm JW. Observations from Mount Oread. In: Wright RA, Greenberg J, Brehm SS, eds. Motivational Analyses of Social Behavior: Building on Jack Brehm’s Contributions to Psychology. Mahwah, NJ: Erlbaum; 2004:247–256.

100 29. Loevinger J. A systemic approach to the construction and evaluation of tests of ability. Psychol Monogr. 1947;61:1–49. 30. World Health Organization. Measuring Quality of Life – The World Health Organization Quality of life Instruments. WHO/MSA/MNH/PSF. Geneva, Switzerland: WHO; 1997:1–15. 31. Scientific Advisory Committee of Medical Outcomes Trust. Assessing health status and quality of life instruments: attributes and review criteria. Qual Life Res. 2002;11:193–205. 32. De Jong FI, Cornelis BE, Wuyts FL, Kooijman PGC, Schutte HK, Oudes MJ, Graamans K. A psychological cascade model for persisting voice problems in teachers. Folia Phoniatr Logop. 2003;55:91–101. 33. Vanhoudt I, Thomas G, Wellens WAR, Vertommen H, De Jong FI. The background biopsychosocial status of teachers with voice problems. J Psychosom Res. 2008;65:371–380.

Journal of Voice, Vol. 29, No. 1, 2015 34. Wright RA, Pantaleo G. Effort processes in achieving performance outcomes: interrelations among and roles of core constructs. Behav Brain Sci. 2013;36:705–706. 35. Pantaleo G, Miron A, Frankowski S, Ferguson, M. Effects of deterrence on intensity of group identification and efforts to protect group identity. Motivation & Emotion, Special Issue on Effort, ISSN: 0146–7239 EISSN: 1573-6644, in press. 36. Lamperti E, Pantaleo G, Finocchiaro CY, Silvani A, Botturi A, Gaviani P, Sarno L, Salmaggi A. Recurrent brain tumor: the impact of illness on patient’s life. Support Care Cancer. 2012;20:1327–1332. 37. Castelli R, Lambertenghi Deliliers G, Colombo R, Moreo G, Gallipoli P, Pantaleo G. Biosimilar epoetin in elderly patients with low-risk myelodysplastic syndromes improves anemia, quality of life, and brain function. Ann Hematol, ISSN: 0939–5555 EISSN: 1432–0584, published online 08 April 2014, http://dx.doi.org/10.1007/s00277-014-2070-8