PVT metrics summarized
J Sleep Res. (2015) 24, 702–713
A new likelihood ratio metric for the psychomotor vigilance test and its sensitivity to sleep loss M A T H I A S B A S N E R , S A R A H M C G U I R E , N A M N I G O E L , H E N G Y I R A O and DAVID F. DINGES Division of Sleep and Chronobiology, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
Keywords cognition, cognitive performance Correspondence Mathias Basner MD, PhD, MSc, Division of Sleep and Chronobiology, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 1019 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104-6021, USA. Tel.: +215 573 5866; fax: +215 573 6410; e-mail [email protected] Accepted in revised form 5 June 2015; received 7 January 2015 DOI: 10.1111/jsr.12322
SUMMARY
The Psychomotor Vigilance Test (PVT) is a widely used assay of behavioural alertness sensitive to the effects of sleep loss and circadian misalignment. However, there is currently no accepted PVT composite outcome metric that captures response slowing, attentional lapses and compensatory premature reactions observed typically in sleep-deprived subjects. We developed a novel likelihood ratio metric (LRM) based on relative frequency distributions in 50 categories of reaction times (RT) and false starts in alert and sleep-deprived subjects (acute total sleep deprivation: n = 31 subjects). The LRM had the largest effect size both in a 33-h total sleep deprivation protocol [1.96; 95% confidence interval (CI): 1.61–2.44; followed by response speed 1/RT, effect size 1.93, 95% CI: 1.55–2.65] and in a chronic partial sleep restriction protocol (1.22; 95% CI: 0.96–1.59; followed by response speed 1/RT, effect size 1.21, 95% CI: 0.94–1.59; 5 nights at 4 h sleep per night; n = 43 subjects). LRM scores correlated highly with response speed (R2 = 0.986), and less well with five other common PVT outcome metrics (R2 = 0.111–0.886). In conclusion, the new LRM is a sensitive PVT outcome metric with high statistical power that takes subtle sleep loss-related changes in the distribution of reaction times (including false starts) into account, is not prone to outliers, does not require baseline data and can be calculated and interpreted easily. Congruence between LRM and PVT response speed and their similar effect size rankings support the use of response speed as the primary, most sensitive and most parsimonious standard PVT outcome metric for determining neurobehavioural deficits from sleep loss.
INTRODUCTION Sleep is a biological imperative, and both sufficient sleep duration and continuity are prerequisites for the restoration of neurobehavioural performance capacity (Banks and Dinges, 2007). Acute total sleep loss, chronic sleep restriction, sleep fragmentation, e.g. through obstructive sleep apnea (Finkel et al., 2009) or noise (Basner et al., 2011b) and circadian misalignment, e.g. through shift work or jet lag (Sack et al., 2007), are prevalent and under-recognized in the population. They increase the risk of sleepiness-related errors and accidents (Barger et al., 2005; Basner et al., 2008; Dinges, 1995), such that every year more than 80 000 automobile crashes in the United States involve drowsy
702
driving (US Department of Transportation, National Highway Traffic Safety Administration, 2011). Representative surveys suggest that 35–40% of the adult US population sleep fewer than 7 h on weekday nights, while approximately 15% of respondents report sleeping fewer than 6 h (Centers for Disease and Prevention, 2011). At the same time, sleepdeprived subjects have been shown to be unable to assess their degree of impairment reliably (Van Dongen et al., 2003, 2011), especially during the biological night (Zhou et al., 2012). This stresses the need for brief, validated and objective measures of cognitive performance that are sensitive to the effects of acute total and chronic partial sleep loss and could potentially be used as fatigue prediction or detection technologies within fatigue risk manageª 2015 European Sleep Research Society
PVT likelihood ratio metric ment systems (Basner and Rubinstein, 2011; Dawson et al., 2014). Among the most reliable effects of sleep deprivation (SDP) is the degradation of attention (Goel et al., 2009; Lim and Dinges, 2010), especially vigilant attention (Dorrian et al., 2005; Lim and Dinges, 2008). The psychomotor vigilance test (PVT) records reaction times (RT) to visual stimuli that occur at random 2–10-s interstimulus intervals over a 10-min period (Basner and Dinges, 2011; Dinges and Kribbs, 1991; Dinges and Powell, 1985; Dorrian et al., 2005; Warm et al., 2008). It has been shown to be a very sensitive measure of vigilant attention and the effects of acute and chronic sleep loss and circadian misalignment (Balkin et al., 2004; Basner and Dinges, 2011; Goel et al., 2009). PVT performance also has relevance for many real-world risks, as sustained attention deficits, slow reactions and impulsive behaviour affect many applied tasks adversely (e.g. all transportation modes and many security-related or industrial tasks) (Basner and Rubinstein, 2011; Dinges, 1995; Gunzelmann et al., 2008; Jongen et al., 2015; Philip and Akerstedt, 2006; Van Dongen and Dinges, 2005). The PVT has negligible aptitude and learning effects, and is therefore probably the most widely used measure of behavioural alertness (Basner and Dinges, 2011; Dorrian et al., 2005; Lim and Dinges, 2008). It
703
is often used as a ‘gold standard’ measure for the neurobehavioural effects of sleep loss, against which other biomarkers or fatigue detection technologies are compared (Chua et al., 2012; Dawson et al., 2014). Sleep deprivation induces robust and reliable deficits in vigilant attention and PVT performance, including an overall slowing of RTs, an increase in the number of errors of omission (i.e. lapses of attention, usually defined as RTs ≥ 500 ms) and a more modest increase in errors of commission (i.e. reactions without a stimulus, or false starts) (Dinges and Mallis, 1998; Van Dongen et al., 2003). However, there is currently no accepted outcome metric that captures all these effects at the same time. For example, attentional lapses capture the sporadically occurring long RTs (or micro-sleeps) that are a hallmark of sleep lossinduced decrements in vigilant attention and wake-state instability, but they do not reflect response slowing, nor do they account for the compensatory increase in the number of premature reactions (i.e. false starts). In fact, by reducing RTs to a binary metric, much information is lost. Except for the false start metric itself, premature reactions are usually not taken into account. They are considered ‘invalid reactions’ and do not contribute to the calculation of existing PVT metrics, even though they increase with SDP (see Fig. 1)
(a) 7%
Relative frequency
6%
Sleep deprived Non-sleep deprived
5% 4% 3% 2%
0%
FS 119 139 159 179 199 209 219 229 239 249 259 269 279 289 299 309 319 329 339 349 359 369 379 389 399 409 419 429 439 449 459 479 499 519 539 559 579 599 649 699 799 899 999 1249 1499 1999 2999 4999 30000
1%
Upper reaction time boundary of category [ms]
ª 2015 European Sleep Research Society
5.0 4.0 3.0 2.0 1.0 0.0
–1.0 –2.0
FS 119 139 159 179 199 209 219 229 239 249 259 269 279 289 299 309 319 329 339 349 359 369 379 389 399 409 419 429 439 449 459 479 499 519 539 559 579 599 649 699 799 899 999 1249 1499 1999 2999 4999 30000
Logarithmized likelihood ratio
(b) Figure 1. Relative frequency distribution of false starts (FS) and reaction times in 49 reaction time categories for PVT bouts performed
J Sleep Res. (2015) 24, 702–713
A new likelihood ratio metric for the psychomotor vigilance test and its sensitivity to sleep loss M A T H I A S B A S N E R , S A R A H M C G U I R E , N A M N I G O E L , H E N G Y I R A O and DAVID F. DINGES Division of Sleep and Chronobiology, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA, USA
Keywords cognition, cognitive performance Correspondence Mathias Basner MD, PhD, MSc, Division of Sleep and Chronobiology, Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, 1019 Blockley Hall, 423 Guardian Drive, Philadelphia, PA 19104-6021, USA. Tel.: +215 573 5866; fax: +215 573 6410; e-mail [email protected] Accepted in revised form 5 June 2015; received 7 January 2015 DOI: 10.1111/jsr.12322
SUMMARY
The Psychomotor Vigilance Test (PVT) is a widely used assay of behavioural alertness sensitive to the effects of sleep loss and circadian misalignment. However, there is currently no accepted PVT composite outcome metric that captures response slowing, attentional lapses and compensatory premature reactions observed typically in sleep-deprived subjects. We developed a novel likelihood ratio metric (LRM) based on relative frequency distributions in 50 categories of reaction times (RT) and false starts in alert and sleep-deprived subjects (acute total sleep deprivation: n = 31 subjects). The LRM had the largest effect size both in a 33-h total sleep deprivation protocol [1.96; 95% confidence interval (CI): 1.61–2.44; followed by response speed 1/RT, effect size 1.93, 95% CI: 1.55–2.65] and in a chronic partial sleep restriction protocol (1.22; 95% CI: 0.96–1.59; followed by response speed 1/RT, effect size 1.21, 95% CI: 0.94–1.59; 5 nights at 4 h sleep per night; n = 43 subjects). LRM scores correlated highly with response speed (R2 = 0.986), and less well with five other common PVT outcome metrics (R2 = 0.111–0.886). In conclusion, the new LRM is a sensitive PVT outcome metric with high statistical power that takes subtle sleep loss-related changes in the distribution of reaction times (including false starts) into account, is not prone to outliers, does not require baseline data and can be calculated and interpreted easily. Congruence between LRM and PVT response speed and their similar effect size rankings support the use of response speed as the primary, most sensitive and most parsimonious standard PVT outcome metric for determining neurobehavioural deficits from sleep loss.
INTRODUCTION Sleep is a biological imperative, and both sufficient sleep duration and continuity are prerequisites for the restoration of neurobehavioural performance capacity (Banks and Dinges, 2007). Acute total sleep loss, chronic sleep restriction, sleep fragmentation, e.g. through obstructive sleep apnea (Finkel et al., 2009) or noise (Basner et al., 2011b) and circadian misalignment, e.g. through shift work or jet lag (Sack et al., 2007), are prevalent and under-recognized in the population. They increase the risk of sleepiness-related errors and accidents (Barger et al., 2005; Basner et al., 2008; Dinges, 1995), such that every year more than 80 000 automobile crashes in the United States involve drowsy
702
driving (US Department of Transportation, National Highway Traffic Safety Administration, 2011). Representative surveys suggest that 35–40% of the adult US population sleep fewer than 7 h on weekday nights, while approximately 15% of respondents report sleeping fewer than 6 h (Centers for Disease and Prevention, 2011). At the same time, sleepdeprived subjects have been shown to be unable to assess their degree of impairment reliably (Van Dongen et al., 2003, 2011), especially during the biological night (Zhou et al., 2012). This stresses the need for brief, validated and objective measures of cognitive performance that are sensitive to the effects of acute total and chronic partial sleep loss and could potentially be used as fatigue prediction or detection technologies within fatigue risk manageª 2015 European Sleep Research Society
PVT likelihood ratio metric ment systems (Basner and Rubinstein, 2011; Dawson et al., 2014). Among the most reliable effects of sleep deprivation (SDP) is the degradation of attention (Goel et al., 2009; Lim and Dinges, 2010), especially vigilant attention (Dorrian et al., 2005; Lim and Dinges, 2008). The psychomotor vigilance test (PVT) records reaction times (RT) to visual stimuli that occur at random 2–10-s interstimulus intervals over a 10-min period (Basner and Dinges, 2011; Dinges and Kribbs, 1991; Dinges and Powell, 1985; Dorrian et al., 2005; Warm et al., 2008). It has been shown to be a very sensitive measure of vigilant attention and the effects of acute and chronic sleep loss and circadian misalignment (Balkin et al., 2004; Basner and Dinges, 2011; Goel et al., 2009). PVT performance also has relevance for many real-world risks, as sustained attention deficits, slow reactions and impulsive behaviour affect many applied tasks adversely (e.g. all transportation modes and many security-related or industrial tasks) (Basner and Rubinstein, 2011; Dinges, 1995; Gunzelmann et al., 2008; Jongen et al., 2015; Philip and Akerstedt, 2006; Van Dongen and Dinges, 2005). The PVT has negligible aptitude and learning effects, and is therefore probably the most widely used measure of behavioural alertness (Basner and Dinges, 2011; Dorrian et al., 2005; Lim and Dinges, 2008). It
703
is often used as a ‘gold standard’ measure for the neurobehavioural effects of sleep loss, against which other biomarkers or fatigue detection technologies are compared (Chua et al., 2012; Dawson et al., 2014). Sleep deprivation induces robust and reliable deficits in vigilant attention and PVT performance, including an overall slowing of RTs, an increase in the number of errors of omission (i.e. lapses of attention, usually defined as RTs ≥ 500 ms) and a more modest increase in errors of commission (i.e. reactions without a stimulus, or false starts) (Dinges and Mallis, 1998; Van Dongen et al., 2003). However, there is currently no accepted outcome metric that captures all these effects at the same time. For example, attentional lapses capture the sporadically occurring long RTs (or micro-sleeps) that are a hallmark of sleep lossinduced decrements in vigilant attention and wake-state instability, but they do not reflect response slowing, nor do they account for the compensatory increase in the number of premature reactions (i.e. false starts). In fact, by reducing RTs to a binary metric, much information is lost. Except for the false start metric itself, premature reactions are usually not taken into account. They are considered ‘invalid reactions’ and do not contribute to the calculation of existing PVT metrics, even though they increase with SDP (see Fig. 1)
(a) 7%
Relative frequency
6%
Sleep deprived Non-sleep deprived
5% 4% 3% 2%
0%
FS 119 139 159 179 199 209 219 229 239 249 259 269 279 289 299 309 319 329 339 349 359 369 379 389 399 409 419 429 439 449 459 479 499 519 539 559 579 599 649 699 799 899 999 1249 1499 1999 2999 4999 30000
1%
Upper reaction time boundary of category [ms]
ª 2015 European Sleep Research Society
5.0 4.0 3.0 2.0 1.0 0.0
–1.0 –2.0
FS 119 139 159 179 199 209 219 229 239 249 259 269 279 289 299 309 319 329 339 349 359 369 379 389 399 409 419 429 439 449 459 479 499 519 539 559 579 599 649 699 799 899 999 1249 1499 1999 2999 4999 30000
Logarithmized likelihood ratio
(b) Figure 1. Relative frequency distribution of false starts (FS) and reaction times in 49 reaction time categories for PVT bouts performed
