Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
Real-time modulation of perceptual eye dominance in humans Jiawei Zhou, Alexandre Reynaud and Robert F. Hess
rspb.royalsocietypublishing.org
Research Cite this article: Zhou J, Reynaud A, Hess RF. 2014 Real-time modulation of perceptual eye dominance in humans. Proc. R. Soc. B 281: 20141717. http://dx.doi.org/10.1098/rspb.2014.1717
Received: 9 July 2014 Accepted: 1 September 2014
Subject Areas: behaviour, cognition, neuroscience Keywords: ocular dominance, perceptual eye dominance, neural plasticity, low-pass, nature movies, movie-watching
Author for correspondence: Robert F. Hess e-mail: [email protected]
McGill Vision Research, Department of Ophthalmology, McGill University, Montreal, Province of Quebec, Canada Ocular dominance (OD) has long served as the model for neural plasticity. The shift of OD has been demonstrated by monocular deprivation in animals only during early visual development. Here, for the first time, we show that perceptual eye dominance can be modulated in real time in normal human adults by varying the spatial image content of movies seen dichoptically by the two eyes over a period as short as 2.5 h. Unlike OD shifts seen in early visual development, this modulation in human eye dominance is not simply a consequence of reduced interocular correlation (e.g. synchronicity) or overall contrast energy, but due to the amplitude reductions of specific image components in one eye’s view. The spatial properties driving this eye dominance change suggest that the underlying mechanism is binocular but not orientationally selective, therefore uniquely locating it to layer 4 B of area V1.
1. Introduction Hubel & Wiesel [1], in their pioneering work on understanding the neural basis of vision, identified two key features of cortical processing: orientation selectivity and ocular dominance (OD). OD refers to the fact that the information from the two eyes is first combined at the level of the early visual cortex with a distribution of left/right eye weights, the so-called OD distribution [2]. This distribution is the outcome of a competitive interaction early in visual development and has provided an important test bed for neuroplastic changes across animal species. OD plasticity can result in such an imbalance that the visual system is capable of processing the information from only one of its two monocular inputs, binocular vision is lost and amblyopia ensues [3,4]. This has been well studied in primates, cats, ferrets and tadpoles, and both the overall contrast energy and synchronicity of the monocular visual inputs have been shown to be important [5]. There is also emerging evidence that the adult brain retains a degree of plasticity throughout life [6–8]; however, little is known of the underlying cortical mechanisms. It has previously been shown that a degree of eye dominance plasticity occurs in human adults if one eye is patched for 150 min [9,10]. Here, we examine the mechanisms underlying these eye dominance changes in the adult by examining its spatial determinants. Subjects view movies presented dichoptically for which the spatial content has been altered in one eye’s view. We show that eye dominance can be modulated in real time while viewing natural images and that these neuroplastic changes operate at the earliest level of cortical processing. These adult neuroplastic changes, unlike their counterparts during early visual development, are not driven by either changes in interocular correlation (i.e. synchronicity) or overall contrast energy but rather by the amplitude of a frequency band that is least represented in natural images, namely non-oriented, high-spatial-frequency components [11].
2. Material and methods (a) Design The visual effect induced by dichoptic viewing of movies, in which one eye’s image was filtered, was tested in three consecutive stages (figure 1a): a pre-movie measurement of binocular sensory balance, a 150 min movie-viewing stage and a post-movie measurement of binocular sensory balance (four sessions immediately after the movie-watching
& 2014 The Author(s) Published by the Royal Society. All rights reserved.
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
(a)
2
pre-movie measure (baseline)
t6
rspb.royalsocietypublishing.org
t0 t 3
t9
post-movie measure
watch movie (2.5 h) t30
RE RE = LE phase = 0°
phase = –22.5°
+
reference line
LE phase = +22.5°
RE > LE phase < 0°
Figure 1. Experimental design. (a) The visual effect induced by dichoptic viewing of monocularly filtered movies was tested in three consecutive stages: a pre-movie measurement of binocular sensory balance, a 150 min movie-viewing stage and a post-movie measurement of binocular sensory balance (four sessions immediately after the movie-watching stage and one session 30 min after the movie-watching stage; each test session lasted 3 min). (b) The method we used for quantifying the eye dominance. Two horizontal sine-wave gratings (0.3 cycle per degree, 6.68 6.68, contrast ¼ 100%) with phase-shifts in opposite directions of the same magnitude (22.58) were presented dichoptically. The perceived phase of the grating after binocular combination depends on the internal weighting of these two inputs. Therefore, any variations of the binocular sensory balance can be quantified by the change in the binocularly perceived phase. (Online version in colour.) stage and one session 30 min after the movie-watching stage; each test session lasted 3 min). Six unilateral filtered movies were generated and tested on different days; they were phase-scrambled (movie S2), 60% root mean square (RMS) contrast (movie S3), 20% RMS contrast (movie S4), low-pass (movie S5), high-pass (movie S6) and horizontal-pass (movie S7) movies. During the movie-viewing stage, subjects watched one of the above-mentioned filtered movies through one eye and a normal version of the same movie through the other eye. For each subject, the eye that watched the filtered movie was randomly selected and fixed across all the movies. Subjects also participated in a normal movie condition (movie S1), in which the two eyes watched the same identical movie.
(b) Apparatus The movie-viewing system was an autonomous handheld system, and could be played everywhere out of the laboratory. Movies were played with an Apple iPod 3 and displayed with a Vuzix Wrap 920 Video Eyewear (Vuzix Corporation, Rochester, New York, NY). The display size was of 318 diagonal and the resolution of each screen was 640 480 pixels, so the image was horizontally stretched to fit the screen. Stimuli for measuring binocular combination were generated and controlled by a Mac computer running MATLAB (MathWorks, Natick, MA) with psychophysics toolbox. The stimuli were dichoptically presented with Z800 pro goggles (eMagin Corporation, Washington, DC), which had a simulated viewing distance of 3.6 m, a spatial resolution of 800 600 pixels, a refresh rate of 60 Hz and a mean luminance of 160 cd m22 in each eye. Both the Vuzix and the Z800 pro goggles have a linear luminance response.
(c) Participants Five adults (age: 23–35 years old; three females), with normal binocular vision (Randot stereo test, 40 arc seconds) participated in the study. Except for the first and second authors, all observers were
naive to the purpose of the experiment. For the three naive subjects, we conducted the experiment in a double-blind manner, in which the first author took charge of the measurement of binocular sensory balance without knowing which movie had been watched, whereas the second author set up the movies for all subjects.
(d) Procedures (i) Measuring the binocular sensory balance Two horizontal sine-wave gratings (0.3 cycles per degree, 6.68 6.68, contrast ¼ 100%) with phase-shifts in opposite directions of the same magnitude (22.58) were presented dichoptically. The perceived phase of the grating after binocular combination depends on the internal weighting of these two inputs. Therefore, any variations of the binocular sensory balance can be quantified by the change in the binocularly perceived phase (figure 1b). To account for any potential positional bias, two configurations were used for assessing the perceived phase: in configuration 1, the phase-shift was þ22.58 in the eye that watched the normal movie and 222.58 in the eye that watched the filtered movie; in configuration 2, the phase-shift was 222.58 in the eye that watched the normal movie and þ22.58 in the eye that watched the filtered movie. The perceived phase for each condition was then quantified by the perceived phase difference between configuration 1 and configuration 2: phase_config.1 2 phase_config.2. Therefore, an increase of the perceived phase (i.e. more positive) after the movie-watching indicates an enhanced contribution of the eye that watched the normal movie, whereas a decrease of the perceived phase (i.e. more negative) after the movie-watching indicates a shift of eye dominance towards the eye that watched the filtered movie. In a typical measurement session, each configuration was measured eight times using the method of constant stimuli. In all, there were 16 trials randomly interleaved for one measurement (two configurations eight repetitions). A short duration of practice was provided for each subject before the start of the
Proc. R. Soc. B 281: 20141717
(b)
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
normal
3
(a)
(b) t3
t6
t9
t30
baseline t0
t6
t9
t30
0
phase-scrambled
0
–5
–5
horizontal-pass –10
–10
n=3 –15
n=5 –15
Figure 2. The effect of unilateral phase scrambling and horizontal filtering. The eye dominance as reflected in the dichoptic phase task is plotted as a function of time. All data points are normalized to the pre-movie-viewing binocularly perceived phase; t0, t3, t6, t9 and t30 refer to time intervals that were measured at 0, 3, 6, 9 and 30 min after subjects finished the 2.5 h movie-viewing. Each time interval was finished in 3 min. In this plot, data points below the zero line indicate a shift of eye dominance towards the eye that views the filtered movie. Three dichoptic movie presentations are illustrated in the inset and their corresponding data are colour-coded. In (a), one eye’s view has been phase-scrambled and there is no significant effect across the times points t0 – t30. Three observers participated in this condition. In (b), one eye’s view has been filtered to only allow horizontal patterns to be represented, and there is a significant effect across the time points t0 – t30. All five observers participated in this condition. The error bars represent the s.e. across observers. (Online version in colour.) experiment to ensure that a measurement could be completed in 3 min and to ensure that performance was stable before movie-watching. The procedure for measuring perceived phase was similar to that reported in previous studies [12], in which observers were asked to adjust the vertical position of a one-pixel reference line to indicate the perceived phase of the binocularly perceived horizontal grating, defined as the location of the centre of the dark stripe of the grating. The initial position of the reference line was randomly (29 to 10 pixels) positioned relative to the centre of the screen in each trial. The reference line was moved with a fixed step size of one pixel, corresponding to 48 phase angle of the sine-wave grating.
(ii) Movies encoding Movies were encoded with OPENSHOT video editor (OpenShot Studios, LLC) in 640 480 pixels resolution, 4 : 3 ratio, 30 frames per second, mp4 format with mpeg4 video codec at 5.00 Mb s21. The two images for the two eyes were encoded in a side-by-side format, so the image resolution was 320 480 in each eye (e.g. normal condition movie S1).
(iii) Movies processing The phase scrambling (movie S2) was performed with a custom MATLAB program using the psychophysics toolbox [13–15]. A random value was added to the phase of each pixel in the Fourier domain. This value was the same for each RGB component and was constant over the whole course of the movie. For each frame, the luminance of the scrambled image was then scaled between the minimum and the maximum of the original image. Thus, the phase scrambling only affected the spatial content, without affecting the temporal properties of the movie. The other filtering operations were processed with OPENSHOT video editor using the MLT framework (Meltytech, LLC). The
low-contrast conditions (movie S3, 60% of contrast; movie S4, 20% of contrast) were processed with the ‘contrast’ filter plugin. The low-pass condition (movie S5) was processed with the ‘blur’ filter with a square kernel of 50 pixels half-size; the resulting RMS contrast was then of approximately 60% (56.30 + 3.02). The high-pass (movie S6) condition was processed with the ‘Sobel’ filter and equated to 60% contrast (59.81 + 1.24). The horizontal-pass (movie S7) condition was processed with a custom filter, based on the ‘BoxBlur’ filter of MLT with a rectangular kernel of 50 pixels horizontal and one pixel vertical half-sizes; the resulting contrast was then approximately 50% (48.85 + 2.26).
3. Results and discussion We have previously shown that this alteration in eye dominance was robust, occurring for local (spatial phase and contrast) as well as for global processing (global motion) [16]. Furthermore, because comparable results can be obtained from an opaque and translucent occluder [16], it is not contingent on a change in the overall light level. This leads to the question, what spatial components in the image drive this form of neuroplastic change in the adult and what is its neural locus? Figure 2 illustrates that these neuroplastic changes are occurring at a low level in the cortical processing of visual information, certainly before orientation selectivity, contour extraction and shape processing. In figure 2a, two eyes see the same movie except that the information displayed to one eye is phase-scrambled (scrambled Fourier phase spectrum) such that while it contains all the same spatial information (unaltered Fourier amplitude spectrum) it
Proc. R. Soc. B 281: 20141717
change of the binocular sensory balance
t3
5
rspb.royalsocietypublishing.org
baseline t0 5
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
baseline
t0
t3
t6
t9
normal
t30
4
0 60% C
Proc. R. Soc. B 281: 20141717
change of the binocular sensory balance
rspb.royalsocietypublishing.org
5
–5
20% C –10
n=5 –15
Figure 3. The effect of image contrast. The eye dominance as reflected in the dichoptic phase task is plotted as a function of time. Results are plotted in the same way as described for figure 2. In black, results are shown for viewing identical images in the two eyes. In dark grey, one eye’s view has been reduced to 60% contrast and there is no significant effect across the times points t0 – t30. In light grey, one eye’s view has been reduced to 20% contrast, and there is a significant eye dominance modulation across the time points t0 – t30. The error bars represent the s.e. from the results of five observers. (Online version in colour.) baseline
t0
t3
t6
t9
t30
change of the binocular sensory balance
5
normal
high-pass
60% C
low-pass
0
–5
–10
n=5 –15
Figure 4. The effect of interocular differences in spatial frequency. The eye dominance as reflected in the dichoptic phase task is plotted as a function of time. Results are plotted in the same way as described for figures 2 and 3. The dotted line represents where one eye’s view has been high-pass filtered and there is no significant eye dominance modulation across the times points t0 – t30. The dashed line represents where one eye’s view has been low-pass filtered and there is a significant eye dominance modulation across the time points t0 – t30. The error bars represent the s.e. from the results of five observers. Both the high- and low-pass images have been equated in contrast energy to an overall reduction of 60% (continuous grey line) that is not significantly different from the data for an unfiltered normal control (continuous black line). (Online version in colour.) contains none of the phase-aligned frequencies that are the building blocks of contours and higher-level spatial representations. Eye dominance is quantified by an interocular phase combination task in which the observer judges the spatial
phase of the binocular percept for monocular stimuli of equal and opposite phase-shift (figure 1b). The results are plotted relative to the eye dominance measured before movie-viewing such that results less than unity indicate a
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
(a) filtered image (b) interocular RMS contrast correlation 1
1
0.60 0.56 0.49
(c) effect on eye dominance –0.4
normal
–5.3 –6.3 –7.5
phase-scrambled high-pass 60% C
0.58 0.49 horizontal-pass
–31.9 –33.0
low-pass 20% C
0.27 0.20 0.05
Figure 5. Movie contrast energies (RMS contrast), interocular correlations and effect on eye dominancy shift. In (a), the different movie conditions are ordered in terms of their filtered image RMS contrast (sampled for each movie every six frames and averaged across the set of five movies). In (b), the different movie conditions are ordered in terms of their interocular correlations (sampled for each movie every six frames and averaged across the set of five movies). In (c), the different movie conditions are ordered in terms of their effect on eye dominance modulation, which was quantified as the values of the area under the phase-time curves. Obviously, neither the interocular correlation nor the contrast energy explains eye dominance modulation. (Online version in colour.) energy (figure 5a), yet only the former (figures 4 and 5c) produced eye dominance changes. What is important is the balance in the amplitude of high-spatial-frequency components in the left and right eye’s retinal image. While the underlying mechanism might operate via a release from adaptation, it differs from previously reported cortical adaptation effects [22] by being intrinsically binocular [16] (an increased sensitivity of one eye accompanied by a decreased sensitivity of the other eye) and not being orientationally selective. The site of this modulation is likely to be at the first synapse in layer 4CB, where cells are binocular but non-oriented [18–20], and possibly regulated by downstream contrast gain control mechanisms receiving suppressive inputs from the contralateral eye [23,24].
Ethics statement. Informed consent was obtained prior to the study. This study complied with the Declaration of Helsinki and was approved by the institutional ethics committee of McGill University.
Data accessibility. Data for figures 2– 4: Dryad: doi:10.5061/dryad.t8011. Funding statement. This work is supported by CIHR grant (no. 53346) and NSERC grant (no. 46528-11) to R.F.H. and a RI-MUHC postdoctoral Award to J.Z. J.Z., A.R. and R.F.H. designed research; J.Z. and A.R. performed research; J.Z. and R.F.H. analysed data; J.Z., A.R. and R.F.H wrote the paper.
Competing financial interests. The authors declare no conflict of interest.
References 1.
Hubel DH, Wiesel TN. 1959 Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148, 574–591.
2.
Hubel DH, Wiesel TN. 1962 Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154.
3.
Sengpiel F, Blakemore C. 1996 The neural basis of suppression and amblyopia in strabismus. Eye 10, 250–258. (doi:10.1038/eye.1996.54)
Proc. R. Soc. B 281: 20141717
–24.0
5
rspb.royalsocietypublishing.org
shift in dominance to the eye that views the filtered movie. The results of an unaltered dichoptic version of the same movie are also provided. There is no significant (withinsubjects repeated measure ANOVA; F1,2 ¼ 0.61, p ¼ 0.52) sustained modulatory effect from this manipulation (compared with normal movie-viewing), suggesting that the spatiotemporal information contained in the amplitude spectrum alone is sufficient for eye dominance. Figure 2b illustrates the orientation selectivity of the eye dominance modulatory effect. The movie seen by one eye was filtered to remove all of its vertical components. This manipulation did produce significant (F1,4 ¼ 11.63, p ¼ 0.03) modulation in eye dominance for the horizontally aligned test stimuli, compared with normal movie-viewing, suggesting that the underlying binocular mechanisms are not tuned for orientation. The vast majority of striate neurons [17] in all cortical layers are tuned for orientation and exhibit binocular properties; however, it is known that neurons in layer 4CB of V1 [18– 20] exhibit binocular properties but lack orientation tuning. Only these neurons have the required properties to support this modulation in eye dominance. In the next manipulation, we simply reduced the magnitude of the Fourier amplitude spectrum to assess how important stimulus contrast was in modulating eye dominance. These results, plotted in the same way as previously described for figure 2, are shown in figure 3 for two different monocular contrast levels, 60% and 20%. No significant shift in dominance was found compared with normal high-contrast movie-viewing for the 60% condition (F1,4 ¼ 0.46, p ¼ 0.53). However, there was a significant (F1,4 ¼ 7.41, p ¼ 0.05) shift in eye dominance to the eye that viewed the 20% contrast movie. Finally, we show that not all reductions to the amplitude spectrum are comparable; the eye dominance modulation critically depends on the form of this contrast reduction across spatial frequency. Figure 4 shows results comparing a lowpass alteration in contrast (the higher the spatial frequency, the more the contrast is reduced) to a high-pass reduction in contrast (the lower the frequency, the more the contrast is reduced), both equated for the same overall contrast energy (60% contrast). The high-pass filtering has no significant (F1,4 ¼ 3.16, p ¼ 0.15) effect compared with the normal movie control. However, the low-pass filtering results in a modulation of eye dominance (F1,4 ¼ 56.16, p ¼ 0.002) compared with the normal movie control, illustrating the importance of high spatial frequencies in the amplitude spectrum. These results suggest that eye dominance, even in adults, is under real-time regulation. Importantly, it is not governed by either the interocular correlation (synchronicity) or the interocular contrast energy (figure 5). For example, the conditions in which the interocular correlation (figure 5b) was disrupted the most was the phase-scrambling condition (figure 2a), and this had no effect on modulating eye dominance (figures 2b and 5c) in the adult, unlike its importance in early visual development [21]. Similarly, for the overall contrast energy, the low-pass and high-pass conditions were matched for contrast
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
4.
12.
13. 14.
16.
17.
18.
19. Horton JC, Hocking DR. 1998 Monocular core zones and binocular border strips in primate striate cortex revealed by the contrasting effects of enucleation, eyelid suture, and retinal laser lesions on cytochrome oxidase activity. J. Neurosci. 18, 5433– 5455. 20. Snodderly DM, Gur M. 1995 Organization of striate cortex of alert, trained monkeys (Macaca fascicularis): ongoing activity, stimulus selectivity, and widths of receptive field activating regions. J. Neurophysiol. 74, 2100–2125. 21. Munz M, Gobert D, Schohl A, Poquerusse J, Podgorski K, Spratt P, Ruthazer ES. 2014 Rapid Hebbian axonal remodeling mediated by visual stimulation. Science 344, 904– 909. (doi:10.1126/ science.1251593) 22. Zhang P, Bao M, Kwon M, He S, Engel SA. 2009 Effects of orientation specific visual deprivation measured using altered reality. Curr. Biol. 19, 1956– 1960. (doi:10.1016/j.cub.2009.10.018) 23. Ding J, Sperling G. 2006 A gain-control theory of binocular combination. Proc. Natl Acad. Sci. USA 103, 1141– 1146. (doi:10.1073/pnas.0509629103) 24. Meese TS, Georgeson MA, Baker DH. 2006 Binocular contrast vision at and above threshold. J. Vis 6, 1224– 1243. (doi:10.1167/6.11.7)
6
Proc. R. Soc. B 281: 20141717
15.
cortical cells. J. Opt. Soc. Am. A, Opt. Image Sci. 4, 2379 –2394. (doi:10.1364/JOSAA.4.002379) Huang CB, Zhou J, Lu ZL, Feng L, Zhou Y. 2009 Binocular combination in anisometropic amblyopia. J. Vis. 9, 11 –16. (doi:10.1167/9.3.17) Brainard DH. 1997 The psychophysics toolbox. Spat. Vision 10, 433–436. (doi:10.1163/156856897X00357) Pelli DG. 1997 The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vision 10, 437 –442. (doi:10.1163/1568568 97X00366) Kleiner M, Brainard D, Pelli D. 2007 What’s new in Psychtoolbox-3? In European conference on visual perception, p. 14. Arezzo, Italy: Perception. Zhou J, Clavagnier S, Hess RF. 2013 Short-term monocular deprivation strengthens the patched eye’s contribution to binocular combination. J. Vis. 13, 11. (doi:10.1167/13.5.12) Hubel DH, Wiesel TN. 1977 Ferrier lecture: functional architecture of macaque monkey visual cortex. Proc. R. Soc. Lond. B 198, 1– 59. (doi:10. 1098/rspb.1977.0085) Blakemore C, Garey LJ, Vital-Durand F. 1978 The physiological effects of monocular deprivation and their reversal in the monkey’s visual cortex. J. Physiol. 283, 223–262.
rspb.royalsocietypublishing.org
Wiesel TN, Hubel DH. 1963 Single-cell responses in striate cortex of kittens deprived of vision in one eye. J. Neurophysiol. 26, 1003– 1017. 5. Daw NW. 2006 Visual development. New York, NY: Springer. 6. Calford MB, Chino YM, Das A, Eysel UT, Gilbert CD, Heinen SJ, Kaas JH, Ullman S. 2005 Neuroscience: rewiring the adult brain. Nature 438, E3; discussion E3 –4. (doi:10.1038/nature04359) 7. Thompson B, Mansouri B, Koski L, Hess RF. 2008 Brain plasticity in the adult: modulation of function in amblyopia with rTMS. Curr. Biol. 18, 1067– 1071. (doi:10.1016/j.cub.2008.06.052) 8. Karni A, Bertini G. 1997 Learning perceptual skills: behavioral probes into adult cortical plasticity. Curr. Opin. Neurobiol. 7, 530–535. (doi:10.1016/S09594388(97)80033-5) 9. Lunghi C, Burr DC, Morrone MC. 2013 Long-term effects of monocular deprivation revealed with binocular rivalry gratings modulated in luminance and in color. J. Vis. 13, 1– 15. (doi:10.1167/13.6.1) 10. Zhou J, Thompson B, Hess RF. 2013 A new form rapid binocular plasticity in adult with amblyopia. Sci. Rep. 3, 2631 –2635. (doi:10.1038/srep02638) 11. Field D. 1987 Relations between the statistics of natural images and the response properties of
Real-time modulation of perceptual eye dominance in humans Jiawei Zhou, Alexandre Reynaud and Robert F. Hess
rspb.royalsocietypublishing.org
Research Cite this article: Zhou J, Reynaud A, Hess RF. 2014 Real-time modulation of perceptual eye dominance in humans. Proc. R. Soc. B 281: 20141717. http://dx.doi.org/10.1098/rspb.2014.1717
Received: 9 July 2014 Accepted: 1 September 2014
Subject Areas: behaviour, cognition, neuroscience Keywords: ocular dominance, perceptual eye dominance, neural plasticity, low-pass, nature movies, movie-watching
Author for correspondence: Robert F. Hess e-mail: [email protected]
McGill Vision Research, Department of Ophthalmology, McGill University, Montreal, Province of Quebec, Canada Ocular dominance (OD) has long served as the model for neural plasticity. The shift of OD has been demonstrated by monocular deprivation in animals only during early visual development. Here, for the first time, we show that perceptual eye dominance can be modulated in real time in normal human adults by varying the spatial image content of movies seen dichoptically by the two eyes over a period as short as 2.5 h. Unlike OD shifts seen in early visual development, this modulation in human eye dominance is not simply a consequence of reduced interocular correlation (e.g. synchronicity) or overall contrast energy, but due to the amplitude reductions of specific image components in one eye’s view. The spatial properties driving this eye dominance change suggest that the underlying mechanism is binocular but not orientationally selective, therefore uniquely locating it to layer 4 B of area V1.
1. Introduction Hubel & Wiesel [1], in their pioneering work on understanding the neural basis of vision, identified two key features of cortical processing: orientation selectivity and ocular dominance (OD). OD refers to the fact that the information from the two eyes is first combined at the level of the early visual cortex with a distribution of left/right eye weights, the so-called OD distribution [2]. This distribution is the outcome of a competitive interaction early in visual development and has provided an important test bed for neuroplastic changes across animal species. OD plasticity can result in such an imbalance that the visual system is capable of processing the information from only one of its two monocular inputs, binocular vision is lost and amblyopia ensues [3,4]. This has been well studied in primates, cats, ferrets and tadpoles, and both the overall contrast energy and synchronicity of the monocular visual inputs have been shown to be important [5]. There is also emerging evidence that the adult brain retains a degree of plasticity throughout life [6–8]; however, little is known of the underlying cortical mechanisms. It has previously been shown that a degree of eye dominance plasticity occurs in human adults if one eye is patched for 150 min [9,10]. Here, we examine the mechanisms underlying these eye dominance changes in the adult by examining its spatial determinants. Subjects view movies presented dichoptically for which the spatial content has been altered in one eye’s view. We show that eye dominance can be modulated in real time while viewing natural images and that these neuroplastic changes operate at the earliest level of cortical processing. These adult neuroplastic changes, unlike their counterparts during early visual development, are not driven by either changes in interocular correlation (i.e. synchronicity) or overall contrast energy but rather by the amplitude of a frequency band that is least represented in natural images, namely non-oriented, high-spatial-frequency components [11].
2. Material and methods (a) Design The visual effect induced by dichoptic viewing of movies, in which one eye’s image was filtered, was tested in three consecutive stages (figure 1a): a pre-movie measurement of binocular sensory balance, a 150 min movie-viewing stage and a post-movie measurement of binocular sensory balance (four sessions immediately after the movie-watching
& 2014 The Author(s) Published by the Royal Society. All rights reserved.
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
(a)
2
pre-movie measure (baseline)
t6
rspb.royalsocietypublishing.org
t0 t 3
t9
post-movie measure
watch movie (2.5 h) t30
RE RE = LE phase = 0°
phase = –22.5°
+
reference line
LE phase = +22.5°
RE > LE phase < 0°
Figure 1. Experimental design. (a) The visual effect induced by dichoptic viewing of monocularly filtered movies was tested in three consecutive stages: a pre-movie measurement of binocular sensory balance, a 150 min movie-viewing stage and a post-movie measurement of binocular sensory balance (four sessions immediately after the movie-watching stage and one session 30 min after the movie-watching stage; each test session lasted 3 min). (b) The method we used for quantifying the eye dominance. Two horizontal sine-wave gratings (0.3 cycle per degree, 6.68 6.68, contrast ¼ 100%) with phase-shifts in opposite directions of the same magnitude (22.58) were presented dichoptically. The perceived phase of the grating after binocular combination depends on the internal weighting of these two inputs. Therefore, any variations of the binocular sensory balance can be quantified by the change in the binocularly perceived phase. (Online version in colour.) stage and one session 30 min after the movie-watching stage; each test session lasted 3 min). Six unilateral filtered movies were generated and tested on different days; they were phase-scrambled (movie S2), 60% root mean square (RMS) contrast (movie S3), 20% RMS contrast (movie S4), low-pass (movie S5), high-pass (movie S6) and horizontal-pass (movie S7) movies. During the movie-viewing stage, subjects watched one of the above-mentioned filtered movies through one eye and a normal version of the same movie through the other eye. For each subject, the eye that watched the filtered movie was randomly selected and fixed across all the movies. Subjects also participated in a normal movie condition (movie S1), in which the two eyes watched the same identical movie.
(b) Apparatus The movie-viewing system was an autonomous handheld system, and could be played everywhere out of the laboratory. Movies were played with an Apple iPod 3 and displayed with a Vuzix Wrap 920 Video Eyewear (Vuzix Corporation, Rochester, New York, NY). The display size was of 318 diagonal and the resolution of each screen was 640 480 pixels, so the image was horizontally stretched to fit the screen. Stimuli for measuring binocular combination were generated and controlled by a Mac computer running MATLAB (MathWorks, Natick, MA) with psychophysics toolbox. The stimuli were dichoptically presented with Z800 pro goggles (eMagin Corporation, Washington, DC), which had a simulated viewing distance of 3.6 m, a spatial resolution of 800 600 pixels, a refresh rate of 60 Hz and a mean luminance of 160 cd m22 in each eye. Both the Vuzix and the Z800 pro goggles have a linear luminance response.
(c) Participants Five adults (age: 23–35 years old; three females), with normal binocular vision (Randot stereo test, 40 arc seconds) participated in the study. Except for the first and second authors, all observers were
naive to the purpose of the experiment. For the three naive subjects, we conducted the experiment in a double-blind manner, in which the first author took charge of the measurement of binocular sensory balance without knowing which movie had been watched, whereas the second author set up the movies for all subjects.
(d) Procedures (i) Measuring the binocular sensory balance Two horizontal sine-wave gratings (0.3 cycles per degree, 6.68 6.68, contrast ¼ 100%) with phase-shifts in opposite directions of the same magnitude (22.58) were presented dichoptically. The perceived phase of the grating after binocular combination depends on the internal weighting of these two inputs. Therefore, any variations of the binocular sensory balance can be quantified by the change in the binocularly perceived phase (figure 1b). To account for any potential positional bias, two configurations were used for assessing the perceived phase: in configuration 1, the phase-shift was þ22.58 in the eye that watched the normal movie and 222.58 in the eye that watched the filtered movie; in configuration 2, the phase-shift was 222.58 in the eye that watched the normal movie and þ22.58 in the eye that watched the filtered movie. The perceived phase for each condition was then quantified by the perceived phase difference between configuration 1 and configuration 2: phase_config.1 2 phase_config.2. Therefore, an increase of the perceived phase (i.e. more positive) after the movie-watching indicates an enhanced contribution of the eye that watched the normal movie, whereas a decrease of the perceived phase (i.e. more negative) after the movie-watching indicates a shift of eye dominance towards the eye that watched the filtered movie. In a typical measurement session, each configuration was measured eight times using the method of constant stimuli. In all, there were 16 trials randomly interleaved for one measurement (two configurations eight repetitions). A short duration of practice was provided for each subject before the start of the
Proc. R. Soc. B 281: 20141717
(b)
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
normal
3
(a)
(b) t3
t6
t9
t30
baseline t0
t6
t9
t30
0
phase-scrambled
0
–5
–5
horizontal-pass –10
–10
n=3 –15
n=5 –15
Figure 2. The effect of unilateral phase scrambling and horizontal filtering. The eye dominance as reflected in the dichoptic phase task is plotted as a function of time. All data points are normalized to the pre-movie-viewing binocularly perceived phase; t0, t3, t6, t9 and t30 refer to time intervals that were measured at 0, 3, 6, 9 and 30 min after subjects finished the 2.5 h movie-viewing. Each time interval was finished in 3 min. In this plot, data points below the zero line indicate a shift of eye dominance towards the eye that views the filtered movie. Three dichoptic movie presentations are illustrated in the inset and their corresponding data are colour-coded. In (a), one eye’s view has been phase-scrambled and there is no significant effect across the times points t0 – t30. Three observers participated in this condition. In (b), one eye’s view has been filtered to only allow horizontal patterns to be represented, and there is a significant effect across the time points t0 – t30. All five observers participated in this condition. The error bars represent the s.e. across observers. (Online version in colour.) experiment to ensure that a measurement could be completed in 3 min and to ensure that performance was stable before movie-watching. The procedure for measuring perceived phase was similar to that reported in previous studies [12], in which observers were asked to adjust the vertical position of a one-pixel reference line to indicate the perceived phase of the binocularly perceived horizontal grating, defined as the location of the centre of the dark stripe of the grating. The initial position of the reference line was randomly (29 to 10 pixels) positioned relative to the centre of the screen in each trial. The reference line was moved with a fixed step size of one pixel, corresponding to 48 phase angle of the sine-wave grating.
(ii) Movies encoding Movies were encoded with OPENSHOT video editor (OpenShot Studios, LLC) in 640 480 pixels resolution, 4 : 3 ratio, 30 frames per second, mp4 format with mpeg4 video codec at 5.00 Mb s21. The two images for the two eyes were encoded in a side-by-side format, so the image resolution was 320 480 in each eye (e.g. normal condition movie S1).
(iii) Movies processing The phase scrambling (movie S2) was performed with a custom MATLAB program using the psychophysics toolbox [13–15]. A random value was added to the phase of each pixel in the Fourier domain. This value was the same for each RGB component and was constant over the whole course of the movie. For each frame, the luminance of the scrambled image was then scaled between the minimum and the maximum of the original image. Thus, the phase scrambling only affected the spatial content, without affecting the temporal properties of the movie. The other filtering operations were processed with OPENSHOT video editor using the MLT framework (Meltytech, LLC). The
low-contrast conditions (movie S3, 60% of contrast; movie S4, 20% of contrast) were processed with the ‘contrast’ filter plugin. The low-pass condition (movie S5) was processed with the ‘blur’ filter with a square kernel of 50 pixels half-size; the resulting RMS contrast was then of approximately 60% (56.30 + 3.02). The high-pass (movie S6) condition was processed with the ‘Sobel’ filter and equated to 60% contrast (59.81 + 1.24). The horizontal-pass (movie S7) condition was processed with a custom filter, based on the ‘BoxBlur’ filter of MLT with a rectangular kernel of 50 pixels horizontal and one pixel vertical half-sizes; the resulting contrast was then approximately 50% (48.85 + 2.26).
3. Results and discussion We have previously shown that this alteration in eye dominance was robust, occurring for local (spatial phase and contrast) as well as for global processing (global motion) [16]. Furthermore, because comparable results can be obtained from an opaque and translucent occluder [16], it is not contingent on a change in the overall light level. This leads to the question, what spatial components in the image drive this form of neuroplastic change in the adult and what is its neural locus? Figure 2 illustrates that these neuroplastic changes are occurring at a low level in the cortical processing of visual information, certainly before orientation selectivity, contour extraction and shape processing. In figure 2a, two eyes see the same movie except that the information displayed to one eye is phase-scrambled (scrambled Fourier phase spectrum) such that while it contains all the same spatial information (unaltered Fourier amplitude spectrum) it
Proc. R. Soc. B 281: 20141717
change of the binocular sensory balance
t3
5
rspb.royalsocietypublishing.org
baseline t0 5
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
baseline
t0
t3
t6
t9
normal
t30
4
0 60% C
Proc. R. Soc. B 281: 20141717
change of the binocular sensory balance
rspb.royalsocietypublishing.org
5
–5
20% C –10
n=5 –15
Figure 3. The effect of image contrast. The eye dominance as reflected in the dichoptic phase task is plotted as a function of time. Results are plotted in the same way as described for figure 2. In black, results are shown for viewing identical images in the two eyes. In dark grey, one eye’s view has been reduced to 60% contrast and there is no significant effect across the times points t0 – t30. In light grey, one eye’s view has been reduced to 20% contrast, and there is a significant eye dominance modulation across the time points t0 – t30. The error bars represent the s.e. from the results of five observers. (Online version in colour.) baseline
t0
t3
t6
t9
t30
change of the binocular sensory balance
5
normal
high-pass
60% C
low-pass
0
–5
–10
n=5 –15
Figure 4. The effect of interocular differences in spatial frequency. The eye dominance as reflected in the dichoptic phase task is plotted as a function of time. Results are plotted in the same way as described for figures 2 and 3. The dotted line represents where one eye’s view has been high-pass filtered and there is no significant eye dominance modulation across the times points t0 – t30. The dashed line represents where one eye’s view has been low-pass filtered and there is a significant eye dominance modulation across the time points t0 – t30. The error bars represent the s.e. from the results of five observers. Both the high- and low-pass images have been equated in contrast energy to an overall reduction of 60% (continuous grey line) that is not significantly different from the data for an unfiltered normal control (continuous black line). (Online version in colour.) contains none of the phase-aligned frequencies that are the building blocks of contours and higher-level spatial representations. Eye dominance is quantified by an interocular phase combination task in which the observer judges the spatial
phase of the binocular percept for monocular stimuli of equal and opposite phase-shift (figure 1b). The results are plotted relative to the eye dominance measured before movie-viewing such that results less than unity indicate a
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
(a) filtered image (b) interocular RMS contrast correlation 1
1
0.60 0.56 0.49
(c) effect on eye dominance –0.4
normal
–5.3 –6.3 –7.5
phase-scrambled high-pass 60% C
0.58 0.49 horizontal-pass
–31.9 –33.0
low-pass 20% C
0.27 0.20 0.05
Figure 5. Movie contrast energies (RMS contrast), interocular correlations and effect on eye dominancy shift. In (a), the different movie conditions are ordered in terms of their filtered image RMS contrast (sampled for each movie every six frames and averaged across the set of five movies). In (b), the different movie conditions are ordered in terms of their interocular correlations (sampled for each movie every six frames and averaged across the set of five movies). In (c), the different movie conditions are ordered in terms of their effect on eye dominance modulation, which was quantified as the values of the area under the phase-time curves. Obviously, neither the interocular correlation nor the contrast energy explains eye dominance modulation. (Online version in colour.) energy (figure 5a), yet only the former (figures 4 and 5c) produced eye dominance changes. What is important is the balance in the amplitude of high-spatial-frequency components in the left and right eye’s retinal image. While the underlying mechanism might operate via a release from adaptation, it differs from previously reported cortical adaptation effects [22] by being intrinsically binocular [16] (an increased sensitivity of one eye accompanied by a decreased sensitivity of the other eye) and not being orientationally selective. The site of this modulation is likely to be at the first synapse in layer 4CB, where cells are binocular but non-oriented [18–20], and possibly regulated by downstream contrast gain control mechanisms receiving suppressive inputs from the contralateral eye [23,24].
Ethics statement. Informed consent was obtained prior to the study. This study complied with the Declaration of Helsinki and was approved by the institutional ethics committee of McGill University.
Data accessibility. Data for figures 2– 4: Dryad: doi:10.5061/dryad.t8011. Funding statement. This work is supported by CIHR grant (no. 53346) and NSERC grant (no. 46528-11) to R.F.H. and a RI-MUHC postdoctoral Award to J.Z. J.Z., A.R. and R.F.H. designed research; J.Z. and A.R. performed research; J.Z. and R.F.H. analysed data; J.Z., A.R. and R.F.H wrote the paper.
Competing financial interests. The authors declare no conflict of interest.
References 1.
Hubel DH, Wiesel TN. 1959 Receptive fields of single neurones in the cat’s striate cortex. J. Physiol. 148, 574–591.
2.
Hubel DH, Wiesel TN. 1962 Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160, 106–154.
3.
Sengpiel F, Blakemore C. 1996 The neural basis of suppression and amblyopia in strabismus. Eye 10, 250–258. (doi:10.1038/eye.1996.54)
Proc. R. Soc. B 281: 20141717
–24.0
5
rspb.royalsocietypublishing.org
shift in dominance to the eye that views the filtered movie. The results of an unaltered dichoptic version of the same movie are also provided. There is no significant (withinsubjects repeated measure ANOVA; F1,2 ¼ 0.61, p ¼ 0.52) sustained modulatory effect from this manipulation (compared with normal movie-viewing), suggesting that the spatiotemporal information contained in the amplitude spectrum alone is sufficient for eye dominance. Figure 2b illustrates the orientation selectivity of the eye dominance modulatory effect. The movie seen by one eye was filtered to remove all of its vertical components. This manipulation did produce significant (F1,4 ¼ 11.63, p ¼ 0.03) modulation in eye dominance for the horizontally aligned test stimuli, compared with normal movie-viewing, suggesting that the underlying binocular mechanisms are not tuned for orientation. The vast majority of striate neurons [17] in all cortical layers are tuned for orientation and exhibit binocular properties; however, it is known that neurons in layer 4CB of V1 [18– 20] exhibit binocular properties but lack orientation tuning. Only these neurons have the required properties to support this modulation in eye dominance. In the next manipulation, we simply reduced the magnitude of the Fourier amplitude spectrum to assess how important stimulus contrast was in modulating eye dominance. These results, plotted in the same way as previously described for figure 2, are shown in figure 3 for two different monocular contrast levels, 60% and 20%. No significant shift in dominance was found compared with normal high-contrast movie-viewing for the 60% condition (F1,4 ¼ 0.46, p ¼ 0.53). However, there was a significant (F1,4 ¼ 7.41, p ¼ 0.05) shift in eye dominance to the eye that viewed the 20% contrast movie. Finally, we show that not all reductions to the amplitude spectrum are comparable; the eye dominance modulation critically depends on the form of this contrast reduction across spatial frequency. Figure 4 shows results comparing a lowpass alteration in contrast (the higher the spatial frequency, the more the contrast is reduced) to a high-pass reduction in contrast (the lower the frequency, the more the contrast is reduced), both equated for the same overall contrast energy (60% contrast). The high-pass filtering has no significant (F1,4 ¼ 3.16, p ¼ 0.15) effect compared with the normal movie control. However, the low-pass filtering results in a modulation of eye dominance (F1,4 ¼ 56.16, p ¼ 0.002) compared with the normal movie control, illustrating the importance of high spatial frequencies in the amplitude spectrum. These results suggest that eye dominance, even in adults, is under real-time regulation. Importantly, it is not governed by either the interocular correlation (synchronicity) or the interocular contrast energy (figure 5). For example, the conditions in which the interocular correlation (figure 5b) was disrupted the most was the phase-scrambling condition (figure 2a), and this had no effect on modulating eye dominance (figures 2b and 5c) in the adult, unlike its importance in early visual development [21]. Similarly, for the overall contrast energy, the low-pass and high-pass conditions were matched for contrast
Downloaded from http://rspb.royalsocietypublishing.org/ on March 29, 2015
4.
12.
13. 14.
16.
17.
18.
19. Horton JC, Hocking DR. 1998 Monocular core zones and binocular border strips in primate striate cortex revealed by the contrasting effects of enucleation, eyelid suture, and retinal laser lesions on cytochrome oxidase activity. J. Neurosci. 18, 5433– 5455. 20. Snodderly DM, Gur M. 1995 Organization of striate cortex of alert, trained monkeys (Macaca fascicularis): ongoing activity, stimulus selectivity, and widths of receptive field activating regions. J. Neurophysiol. 74, 2100–2125. 21. Munz M, Gobert D, Schohl A, Poquerusse J, Podgorski K, Spratt P, Ruthazer ES. 2014 Rapid Hebbian axonal remodeling mediated by visual stimulation. Science 344, 904– 909. (doi:10.1126/ science.1251593) 22. Zhang P, Bao M, Kwon M, He S, Engel SA. 2009 Effects of orientation specific visual deprivation measured using altered reality. Curr. Biol. 19, 1956– 1960. (doi:10.1016/j.cub.2009.10.018) 23. Ding J, Sperling G. 2006 A gain-control theory of binocular combination. Proc. Natl Acad. Sci. USA 103, 1141– 1146. (doi:10.1073/pnas.0509629103) 24. Meese TS, Georgeson MA, Baker DH. 2006 Binocular contrast vision at and above threshold. J. Vis 6, 1224– 1243. (doi:10.1167/6.11.7)
6
Proc. R. Soc. B 281: 20141717
15.
cortical cells. J. Opt. Soc. Am. A, Opt. Image Sci. 4, 2379 –2394. (doi:10.1364/JOSAA.4.002379) Huang CB, Zhou J, Lu ZL, Feng L, Zhou Y. 2009 Binocular combination in anisometropic amblyopia. J. Vis. 9, 11 –16. (doi:10.1167/9.3.17) Brainard DH. 1997 The psychophysics toolbox. Spat. Vision 10, 433–436. (doi:10.1163/156856897X00357) Pelli DG. 1997 The VideoToolbox software for visual psychophysics: transforming numbers into movies. Spat. Vision 10, 437 –442. (doi:10.1163/1568568 97X00366) Kleiner M, Brainard D, Pelli D. 2007 What’s new in Psychtoolbox-3? In European conference on visual perception, p. 14. Arezzo, Italy: Perception. Zhou J, Clavagnier S, Hess RF. 2013 Short-term monocular deprivation strengthens the patched eye’s contribution to binocular combination. J. Vis. 13, 11. (doi:10.1167/13.5.12) Hubel DH, Wiesel TN. 1977 Ferrier lecture: functional architecture of macaque monkey visual cortex. Proc. R. Soc. Lond. B 198, 1– 59. (doi:10. 1098/rspb.1977.0085) Blakemore C, Garey LJ, Vital-Durand F. 1978 The physiological effects of monocular deprivation and their reversal in the monkey’s visual cortex. J. Physiol. 283, 223–262.
rspb.royalsocietypublishing.org
Wiesel TN, Hubel DH. 1963 Single-cell responses in striate cortex of kittens deprived of vision in one eye. J. Neurophysiol. 26, 1003– 1017. 5. Daw NW. 2006 Visual development. New York, NY: Springer. 6. Calford MB, Chino YM, Das A, Eysel UT, Gilbert CD, Heinen SJ, Kaas JH, Ullman S. 2005 Neuroscience: rewiring the adult brain. Nature 438, E3; discussion E3 –4. (doi:10.1038/nature04359) 7. Thompson B, Mansouri B, Koski L, Hess RF. 2008 Brain plasticity in the adult: modulation of function in amblyopia with rTMS. Curr. Biol. 18, 1067– 1071. (doi:10.1016/j.cub.2008.06.052) 8. Karni A, Bertini G. 1997 Learning perceptual skills: behavioral probes into adult cortical plasticity. Curr. Opin. Neurobiol. 7, 530–535. (doi:10.1016/S09594388(97)80033-5) 9. Lunghi C, Burr DC, Morrone MC. 2013 Long-term effects of monocular deprivation revealed with binocular rivalry gratings modulated in luminance and in color. J. Vis. 13, 1– 15. (doi:10.1167/13.6.1) 10. Zhou J, Thompson B, Hess RF. 2013 A new form rapid binocular plasticity in adult with amblyopia. Sci. Rep. 3, 2631 –2635. (doi:10.1038/srep02638) 11. Field D. 1987 Relations between the statistics of natural images and the response properties of
