Projection-type dual-view three-dimensional display system based on integral imaging Jinsoo Jeong, Chang-Kun Lee, Keehoon Hong, Jiwoon Yeom, and Byoungho Lee* School of Electrical Engineering, Seoul National University, Gwanak-gu Gwanak-ro 1, Seoul 151-744, South Korea *Corresponding author: [email protected] Received 1 May 2014; accepted 5 June 2014; posted 13 June 2014 (Doc. ID 211203); published 22 July 2014
A dual-view display system provides two different images in different directions. Most of them only present two-dimensional images for observers. In this paper, we propose a projection-type dual-view three-dimensional (3D) display system based on integral imaging. To assign directivities to the images, a projection-type display and dual-view screen with lenticular lenses are implemented. The lenticular lenses split the collimated image from the projection device into two different directions. The separated images are integrated by a single lens array in front of the screen, and full-parallax 3D images are observed in two different viewing regions. The visibility of the reconstructed 3D images can be improved by using high-density lenticular lenses and a high numerical aperture lens array. We explain the principle of the proposed method and verify the feasibility of the proposed system with simulations and experimental results. © 2014 Optical Society of America OCIS codes: (110.0110) Imaging systems; (110.2990) Image formation theory; (100.6890) Threedimensional image processing. http://dx.doi.org/10.1364/AO.53.000G12
1. Introduction
Dual-view display systems can provide two distinct images to different observers. This property can be used for various applications. For example, the dual-view display system in an automobile represents traffic information for a driver and news for a passenger simultaneously. In the medical field, a doctor can learn the state of a patient, and an assistant can check other information during a surgery. Dual-view display systems are very similar to three-dimensional (3D) display systems in terms of generating two or more distinguished views [1]. The dual-view display system shows distinguished views to different observers, while the 3D display system shows perspectives with binocular disparity to the different eyes of an observer. With this similarity, the basic principles of the dual-view display system and 3D display system are tied together closely. 1559-128X/14/270G12-07$15.00/0 © 2014 Optical Society of America G12
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
By using polarization glasses or liquid-crystal shutter glasses, we can easily implement the dual-view display systems or the 3D display systems [1–4]. Also, glasses-free dual-view display systems and 3D display systems can be implemented by using optical components such as lenticular lenses, parallax barriers, and a lens array [5–7]. They can generate two or more views using the spatial multiplexing method or time multiplexing method [1]. Dual-view display systems providing two-dimensional images have been commercialized, but most of them cannot provide 3D images. The resolution of displaying devices such as flat panel displays (FPDs) was not enough to implement dual view and 3D simultaneously because both of them use the spatial multiplexing method. However, there are many advantages if dual-view display systems can present 3D images. For example, 3D images can let a driver acquire traffic information without changing his focus from the outdoor scene to the display panel, and it can provide augmented reality to a doctor for a precise surgery. Recently, the
development of ultrahigh-resolution FPD and spatial light modulators has increased the quality of 3D displays [8,9]. It can make it possible to combine the dual-view and 3D display system with highresolution 3D images. Several studies related to dual-view 3D display systems have been reported [10–12]. These studies apply the integral imaging method to present 3D images. Integral imaging uses a lens array to capture 3D objects as elemental images and to reconstruct 3D images for the elemental images. It is one of the promising techniques for producing 3D images because it has the advantages of quasi-continuous viewpoints and full-parallax 3D images that cause low eye fatigue [7,9,13]. Because of these fine characteristics, studies on integral imaging have been continued vigorously since they were first introduced, and these characteristics can make the dual-view display systems provide 3D images appropriately. Generating multiple viewing zones using integral imaging, which is one of the research areas related to the dual-view and 3D display system, was proposed by Choi et al. [10]. It uses a dynamic barrier array to generate multiple viewing zones for 3D images. The viewing zones generated by dynamic barriers can be used for the viewing zones of the dual-view 3D display system. But it is hard to implement dynamically moving barriers for practical use. Recently, dual-view 3D display systems based on integral imaging were introduced by Wu and coworkers [11,12]. The system generates two different viewing zones by using the spatial multiplex technique. The viewing zones of these systems are narrow and not separated completely, because two viewing zones share a viewing zone of the ordinary integral imaging system. They use a polarization parallax barrier to enhance the viewing angle [12]. However, it degrades the resolution and brightness of the 3D images when the polarization barriers block the elemental images. And these systems use a pinhole array instead of a lens array. It causes black grids in the 3D images, which deteriorate the resolution and brightness of the images. For realizing a 3D experience in the dual-view display system, completely separated viewing zones and bright 3D images with high resolution are required. In this paper, we propose a projection-type dualview 3D display based on integral imaging. To separate the images clearly, a high-resolution projection device and lenticular lenses are used. The 3D image in each direction is integrated by a lens array that allows high brightness. In the experiments, we implement the proposed system and observe the two different 3D images in each viewing zone. 2. Principle A.
Concept of the Proposed System
Figure 1 shows the schematic diagram of the proposed system. The projector generates elemental images for the dual-view 3D images. The elemental
Fig. 1. Schematic diagram of the proposed dual-view 3D display system.
images from the projector are parallelized by the collimating lens. When the collimated elemental images are projected on the lenticular lenses, the propagation directions of the projected elemental images are divided into left and right. The lens array placed in front of the lenticular lenses generates a left and right viewing zone from the separated elemental images, and different 3D images are observed in each viewing zone with sharing the lens array. In this configuration, we implement the dualview 3D display based on integral imaging using lenticular lenses and a lens array. A projection-type display is used for the proposed system, because the projection type has significant advantages in collimating the images, aligning them to the lenticular lenses, and implementing high resolution in a small screen [14]. B. Dual-View Generation Using Lenticular Lens
The most important concept of a dual-view display system is how to generate two different views. In the proposed system, lenticular lenses are used for producing a dual-view property. The lenticular lenses are an array of cylindrical concave lenses. Since the normal incidence ray into the lenticular lens passes the focal point, the direction of the ray after passing through the lenticular lens can be adjusted by changing the location of the incidence. These characteristics enable division of the collimated image into different directions. We can calculate the separation angle θsep between the left viewing zone and right viewing zone by using a geometrical relation, as shown in Fig. 2. It is expressed by θsep 2 tan−1
D ; fl
(1)
where D is the distance of the ray from the center of each lenticular lens, which is the location of the incidence, and f l is the focal length of the lenticular lens. Unlike the previous dual-view integral imaging systems [11,12], by changing D and f l , the separation 20 September 2014 / Vol. 53, No. 27 / APPLIED OPTICS
G13
Fig. 2. Splitting elemental images at lenticular lenses.
angle of the proposed system can be adjusted flexibly according to the application, while that of the previous systems is fixed. To divide the whole image from the projector into two different elemental images for left and right viewing zones, the two elemental images should be interwoven to propagate into proper viewing zones. The mapping position of each elemental image is determined by θsep from Eq. (1). The elemental images of the left and right viewing zones are interwoven alternatively line by line, as shown in Fig. 2, while staying a distance D from the center of the lenticular lens. In this case, the maximum number of pixels that one lenticular lens can contain is two, which is the same as the number of the views. If there are more than two pixels in the lenticular lens, they generate more than two views with different separation angles. The extra view causes crosstalk between the viewing zones and makes the 3D images blurred. Instead of increasing the number of pixels in the lenticular lens, if the lenticules per inch (LPI) of the lenticular lenses is higher, the visibility of each view will be improved. C. Elemental Image Generation for Off-Axis Integral Imaging System
In an on-axis integral imaging system, the normal axis of the elemental image surface and the optical axis of the elemental lens are on the same axis. Therefore, when the elemental images are generated, 3D objects are picked up with an on-axis direction, like in Fig. 3(a). The elemental lenses act like cameras, and they capture the disparities of the 3D objects in different directions as elemental images. In Fig. 3(b), the direction of the rays in the dualview system and the surface normal direction of the lens array have an off-axis configuration. The elemental images for the left and right viewing zones should be generated with consideration of the off-axis G14
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
Fig. 3. Elemental image generation method for (a) on-axis integral imaging and (b) off-axis integral imaging.
configuration. The elemental lens of the system acts as a camera, as in conventional on-axis integral imaging. However, the axis that connects the center of the elemental lens and the elemental image is slanted by θsep ∕2, as shown in Fig. 3(b). The lens array captures disparities of the 3D objects obliquely, which means that the elemental images in the off-axis configuration can be generated by shifting the lens array along the horizontal direction. The shifted distance of the elemental images d can be calculated by d g tan
θsep ; 2
(2)
where g is the gap between the lens array and the elemental images. After the elemental images of each viewing zone are generated in the off-axis configuration, the elemental images are interwoven properly to satisfy the dual-view condition as mentioned earlier. Figures 4(a) and 4(b) represent the elemental images
Fig. 5. Viewing angle of (a) on-axis and (b) off-axis integral imaging.
inclined by θsep ∕2. Assuming that all of the elemental lenses in the lens array are used for reconstructing 3D images, the minimum distance between the lens array and the viewing zones L is defined by L m − 1g;
Fig. 4. Computer-generated elemental images for (a) left viewing zone, (b) right viewing zone, and (c) interwoven elemental images.
(5)
where m is the number of the elemental lenses on one side of the lens array. Inside each viewing zone, the left and right elemental images are integrated by sharing the lens array. Since the viewing zones are separated completely without any overlapping zone, the proposed system can display two different 3D images independently without crosstalk. 3. Simulations
of the left and right viewing zones, and Fig. 4(c) represents the interwoven elemental images for the dual-view 3D display system. D.
Viewing Parameters Analysis of the Proposed System
The on-axis integral imaging system reconstructs 3D images on the normal axis, and the viewing angle θvo is simply calculated as shown in Fig. 5(a). It is expressed by tan
θvo p ; 2 2g
The visibility of 3D images in the proposed system is mainly affected by both the LPI of the lenticular lenses and the numerical aperture (NA) of the lens array. To confirm the principles of the proposed system and to evaluate the visibility of the reconstructed 3D images in each viewing zone, we perform optical simulations of the proposed system by using LightTools 8.1, which is an optical simulation tool based on the nonsequential Monte Carlo ray-tracing
(3)
where p is the pitch of the elemental lens. In the proposed system, the separated elemental images are reconstructed on the axis inclined by θsep ∕2, as shown in Fig. 5(b). In the off-axis configuration, the viewing angle θv can be calculated by tan θv
1 1 tan2
θsep 2
g p
p − 4g
:
(4)
Figure 6 shows the schematic diagram of the left and right viewing zones in the proposed system. The viewing zones are constructed along the axes
Fig. 6. Viewing parameters of the proposed system. 20 September 2014 / Vol. 53, No. 27 / APPLIED OPTICS
G15
Fig. 7. Schematic diagram of the optical arrangement in LightTools simulation.
technique. In the simulations, the lenticular lenses separate the propagation directions of the interwoven image into two different directions, and the lens array integrates the elemental images into a 3D image. Two virtual cameras are placed in the left and right viewing zones to observe the reconstructed images in each viewing zone. The resolution of the elemental images used in the simulations is 1920 × 1080 pixels, and their pixel pitch is 0.042 mm. The schematic diagram of the optical arrangements in the simulations is shown in Fig. 7, and the detailed specifications are listed in Table 1. The visibility of the separated elemental images is proportional to the LPI of the lenticular lens. Since the number of lenticular lenses for each elemental lens of the lens array determines the resolution of the elemental images for each viewing zone, the high LPI of lenticular lenses can improve the visibility of the reconstructed 3D images by providing highresolution elemental images. Moreover, since the proposed system is based on off-axis integral imaging, some of the obliquely incident rays cannot pass through the lens array and cannot be integrated into the 3D images. To evaluate these relations, the simulations are performed with varying LPI of the lenticular lenses and the focal length of the lens array, as shown in Fig. 8. In the simulation, the elemental images are generated to present a number “3” at the left viewing zone and a letter “D” at the right viewing zone. The gap between the elemental images and the lens array is adjusted to reconstruct the 3D images at 50 mm in front of the lens array. As shown in Fig. 8(a), the different symbols “3” and “D” can be observed from different viewing positions Table 1.
Specifications of the Simulations
Simulations
1
2
3
Lenticular lenses
LPI Thickness (mm) Focal length (mm)
50 0.61 0.61
100 0.61 0.61
100 0.61 0.61
Lens array
Lens pitch (mm) 5 5 5 Number of lenses 15 × 15 15 × 15 15 × 15 Focal length (mm) 10 10 25
G16
that satisfy the dual-view property of the proposed method. In the reconstructed images, periodic black lines appear between the color information of the 3D images. Since the elemental images are interwoven to generate the dual-view characteristic, the thin black lines come from the separation distance between the pixels that is intended to separate the elemental images into two different directions. The thick black lines appear when some of the rays cannot pass through the elemental lens of the lens array because the NA of the lens array limits the amount of rays. When the rays are obliquely incident on the lens, the limitation by the NA is more severe than the normal incidence. In this simulation, the visibility of the reconstructed 3D images can be evaluated by calculating the maximum number of the color lines in the horizontal direction, and we define the number as a visibility factor. The deteriorated visibility of the reconstructed images, which is caused by the thin black lines, can be improved by using high LPI lenticular lenses and densely interwoven elemental images. By comparing Figs. 8(a) and 8(b), the visibility factors of the 3D images are improved in the simulations by increasing LPI of the lenticular lenses from 50 to 100. In the left viewing zone, the visibility factor of the 3D image is improved from 10 to 14, and in the right viewing zone it is improved from 14 to 20. Since the thick black lines are caused by the limited NA of the lens array, they can be removed by using a
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
Fig. 8. Reconstructed 3D images in the simulation by using lenticular lenses of (a) 50 LPI, (b) 100 LPI, and (c) 100 LPI. The lens arrays in the simulation have focal lengths of (a) 10 mm, (b) 10 mm, and (c) 20 mm.
high NA lens array. The NA of the lens array in the simulation is adjusted by varying the focal length of the lens array while maintaining its pitch. By comparing Figs. 8(b) and 8(c), the visibility factor of the reconstructed 3D images is improved in the simulation by increasing the NA of the lens array. In the left viewing zone, the visibility factor of the 3D images is improved from 14 to 29, and in the right viewing zone, it is improved from 20 to 43. According to the simulation results, the visibility of the dualview 3D images will be improved by using the high LPI lenticular lenses and the high NA lens array. 4. Experiments
To investigate the feasibility of the proposed system, we implement the experimental setup as shown in Fig. 9. For the projection part, we use a Panasonic PT-AE1000 projector and a telecentric lens of Edmund Optics. The elemental images have the resolution of 1920 × 1080, and they are collimated by the telecentric lens. The lenticular lenses give the directivities to the collimated images, and a dual view is generated. We use lenticular lenses of 50 LPI, which means about two lenticules per millimeter. The lens array with 5 mm lens pitch and 10 mm focal length is used. An elemental lens of the lens array contains about 10 lenticules. The pixels of the interwoven elemental images are mapped with uniform distance to fit in the size of the lenticular lens. The separation angle θsep is determined by that distance, according to Eq. (1). In a projection-type imaging system, the exit pupil of the projection optics limits the visible image area and lowers the fill factor when we directly observe the projected image without diffuser. To observe the entire image from the projection-type imaging system, a diffusive screen is required in practice. In the proposed system, a vertical diffuser (PC6001-80, manufactured by Fusion Optix) is placed in front of the lenticular lenses for preserving the directivity of the dual-view system and solving the low fill factor problem due to the exit pupil of the projector. The separated elemental images are integrated by the lens array, and 3D images are reconstructed in each viewing zone. The left viewing zone is formed from −26° to −5° in front of the lens array, and the right viewing zone is formed from 5° to 26° in front of the lens array, according to Eqs. (1) and (4). The detailed specifications of the prototype are presented in Table 2.
Fig. 9. Experimental setup of the proposed system.
Table 2.
Specifications of the Prototype
Elemental images
Resolution (pixels) Pixel pitch (mm)
1920 × 1080 0.042
Lenticular lenses
LPI Thickness (mm) Focal length (mm)
50 0.61 0.61
Vertical diffuser
Diffusing angle (°) Horizontal Vertical
60 1
Lens array
Lens pitch (mm) Number of lenses Focal length (mm)
5×5 15 × 15 10
Viewing parameters
θsep (°) θv (°) g (mm) L (mm)
32 21 12.5 175
To verify that the 3D images of each viewing zone are reconstructed correctly, we use 3D images with four letters placed in different depths. In the left viewing zone, a letter “A” is integrated at 60 mm in front of the lens array and a letter “B” is at 40 mm. In the right viewing zone, a letter “C” is integrated at 60 mm in front of the lens array and a letter “D” is at 40 mm. As shown in Fig. 10, the two different 3D images are reconstructed at each
Fig. 10. Reconstructed 3D images of (a) left viewing zone and (b) right viewing zone. 20 September 2014 / Vol. 53, No. 27 / APPLIED OPTICS
G17
viewing zone. Figure 10(a) presents the reconstructed 3D image of the left viewing zone, and Fig. 10(b) presents the reconstructed 3D images of the right viewing zone. We can find that the relative positions of the two letters are changing with viewing directions in each viewing zone, and the disparities of both vertical and horizontal directions are verified. Since the 3D images are reconstructed through the off-axis configuration, there are distortions of the images by aberrations. It can be improved by applying the predistortion method: the image distortions caused by off-axis configuration can be estimated and compensated during pre-image processing [15].
2.
3.
4.
5.
5. Conclusion
In this paper, we proposed a projection-type dualview 3D display system using integral imaging that provides two completely separated viewing zones and full-parallax 3D images in each viewing zone. The projection image is collimated by the telecentric lens, and the lenticular lenses split the image into two different directions. The lens array generates 3D images in the left and right viewing zones. The characteristics and parameters of the proposed system are analyzed by using the optical simulation. Through the experiment, we confirm that the viewing zones are properly separated, and the perspectives of 3D images from computer-generated elemental images are observed in each viewing zone. The visibility of the 3D images can be improved by using high LPI lenticular lenses and a high NA lens array. We expect that the proposed system can be applied in various areas that have potential demand for the dual-view 3D display system, such as the automotive industry or medical field.
6.
7.
8.
9.
10.
11.
12.
This research was supported by “The CrossMinistry Giga KOREA Project” of The Ministry of Science, ICT and Future Planning, Korea. [GK13D0200, Development of Super Multi-View (SMV) Display Providing Real-Time Interaction].
13.
References
15.
1. J. Hong, Y. Kim, H.-J. Choi, J. Hahn, J.-H. Park, H. Kim, S.-W. Min, N. Chen, and B. Lee, “Three-dimensional display
G18
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
14.
technologies of recent interest: principles, status, and issues,” Appl. Opt. 50, H87–H115 (2011). S. S. Kim, B. H. You, H. Choi, B. H. Berkeley, and N. D. Kim, “World’s first 240 Hz TFT-LCD technology for full-HD LCD-TV and its application to 3D display,” SID Int. Symp. Dig. Tech. Pap. 40, 424–427 (2009). D.-S. Kim, S.-M. Park, J.-H. Jung, and D.-C. Hwang, “New 240 Hz driving method for full HD and high quality 3D LCD TV,” SID Int. Symp. Dig. Tech. Pap. 41, 762–765 (2010). H. Kang, S.-D. Roh, I.-S. Baik, H.-J. Jung, W.-N. Jeong, J.-K. Shin, and I.-J. Chung, “A novel polarizer glasses-type 3D displays with a patterned retarder,” SID Int. Symp. Dig. Tech. Pap. 41, 1–4 (2010). J.-Y. Son, V. V. Saveljev, Y.-J. Choi, J.-E. Bahn, H.-H. Choi, and S.-K. Kim, “Parameters for designing autostereoscopic imaging systems based on lenticular, parallax barrier and integral photography plates,” Opt. Eng. 42, 3326–3333 (2003). J.-Y. Son, V. Saveljev, J.-S. Kim, S.-S. Kim, and B. Javidi, “Viewing zones in three-dimensional imaging systems based on lenticular, parallax-barrier, and microlens-array plates,” Appl. Opt. 43, 4985–4992 (2004). J.-H. Park, K. Hong, and B. Lee, “Recent progress in threedimensional information processing based on integral imaging,” Appl. Opt. 48, H77–H94 (2009). L.-Y. Chen, Y.-Y. Chen, H.-L. Hou, Y. Mo, and C.-Y. Chiu, “Development of the world-wide largest 110′ 4K2K 3D TFTLCD display,” SID Int. Symp. Dig. Tech. Pap. 44, 100–103 (2013). S.-G. Park, J. Yeom, Y. Jeong, N. Chen, J.-Y. Hong, and B. Lee, “Recent issues on integral imaging and its applications,” J. Inf. Disp. 15, 37–46 (2014). H. Choi, S.-W. Min, S. Jung, J.-H. Park, and B. Lee, “Multipleviewing-zone integral imaging using a dynamic barrier array for three-dimensional displays,” Opt. Express 11, 927–932 (2003). F. Wu, H. Deng, C.-G. Luo, D.-H. Li, and Q.-H. Wang, “Dualview integral imaging three-dimensional display,” Appl. Opt. 52, 4911–4914 (2013). F. Wu, Q.-H. Wang, C.-G. Luo, D.-H. Li, and H. Deng, “Dualview integral imaging 3D display using polarizer parallax barriers,” Appl. Opt. 53, 2037–2039 (2014). Y. Kim, J. Kim, K. Hong, H. K. Yang, J.-H. Jung, H. Choi, S.-W. Min, J.-M. Seo, J.-M. Hwang, and B. Lee, “Accommodative response of integral imaging in near distance,” J. Disp. Technol. 8, 70–78 (2012). Y. Kim, K. Hong, J. Yeom, J. Hong, J.-H. Jung, Y. W. Lee, J.-H. Park, and B. Lee, “A frontal projection-type three-dimensional display,” Opt. Express 20, 20130–20138 (2012). Y. M. Kim, K.-M. Jung, and S.-W. Min, “Analysis of off-axis integral floating system using concave mirror,” J. Opt. Soc. Korea 16, 270–276 (2012).
A dual-view display system provides two different images in different directions. Most of them only present two-dimensional images for observers. In this paper, we propose a projection-type dual-view three-dimensional (3D) display system based on integral imaging. To assign directivities to the images, a projection-type display and dual-view screen with lenticular lenses are implemented. The lenticular lenses split the collimated image from the projection device into two different directions. The separated images are integrated by a single lens array in front of the screen, and full-parallax 3D images are observed in two different viewing regions. The visibility of the reconstructed 3D images can be improved by using high-density lenticular lenses and a high numerical aperture lens array. We explain the principle of the proposed method and verify the feasibility of the proposed system with simulations and experimental results. © 2014 Optical Society of America OCIS codes: (110.0110) Imaging systems; (110.2990) Image formation theory; (100.6890) Threedimensional image processing. http://dx.doi.org/10.1364/AO.53.000G12
1. Introduction
Dual-view display systems can provide two distinct images to different observers. This property can be used for various applications. For example, the dual-view display system in an automobile represents traffic information for a driver and news for a passenger simultaneously. In the medical field, a doctor can learn the state of a patient, and an assistant can check other information during a surgery. Dual-view display systems are very similar to three-dimensional (3D) display systems in terms of generating two or more distinguished views [1]. The dual-view display system shows distinguished views to different observers, while the 3D display system shows perspectives with binocular disparity to the different eyes of an observer. With this similarity, the basic principles of the dual-view display system and 3D display system are tied together closely. 1559-128X/14/270G12-07$15.00/0 © 2014 Optical Society of America G12
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
By using polarization glasses or liquid-crystal shutter glasses, we can easily implement the dual-view display systems or the 3D display systems [1–4]. Also, glasses-free dual-view display systems and 3D display systems can be implemented by using optical components such as lenticular lenses, parallax barriers, and a lens array [5–7]. They can generate two or more views using the spatial multiplexing method or time multiplexing method [1]. Dual-view display systems providing two-dimensional images have been commercialized, but most of them cannot provide 3D images. The resolution of displaying devices such as flat panel displays (FPDs) was not enough to implement dual view and 3D simultaneously because both of them use the spatial multiplexing method. However, there are many advantages if dual-view display systems can present 3D images. For example, 3D images can let a driver acquire traffic information without changing his focus from the outdoor scene to the display panel, and it can provide augmented reality to a doctor for a precise surgery. Recently, the
development of ultrahigh-resolution FPD and spatial light modulators has increased the quality of 3D displays [8,9]. It can make it possible to combine the dual-view and 3D display system with highresolution 3D images. Several studies related to dual-view 3D display systems have been reported [10–12]. These studies apply the integral imaging method to present 3D images. Integral imaging uses a lens array to capture 3D objects as elemental images and to reconstruct 3D images for the elemental images. It is one of the promising techniques for producing 3D images because it has the advantages of quasi-continuous viewpoints and full-parallax 3D images that cause low eye fatigue [7,9,13]. Because of these fine characteristics, studies on integral imaging have been continued vigorously since they were first introduced, and these characteristics can make the dual-view display systems provide 3D images appropriately. Generating multiple viewing zones using integral imaging, which is one of the research areas related to the dual-view and 3D display system, was proposed by Choi et al. [10]. It uses a dynamic barrier array to generate multiple viewing zones for 3D images. The viewing zones generated by dynamic barriers can be used for the viewing zones of the dual-view 3D display system. But it is hard to implement dynamically moving barriers for practical use. Recently, dual-view 3D display systems based on integral imaging were introduced by Wu and coworkers [11,12]. The system generates two different viewing zones by using the spatial multiplex technique. The viewing zones of these systems are narrow and not separated completely, because two viewing zones share a viewing zone of the ordinary integral imaging system. They use a polarization parallax barrier to enhance the viewing angle [12]. However, it degrades the resolution and brightness of the 3D images when the polarization barriers block the elemental images. And these systems use a pinhole array instead of a lens array. It causes black grids in the 3D images, which deteriorate the resolution and brightness of the images. For realizing a 3D experience in the dual-view display system, completely separated viewing zones and bright 3D images with high resolution are required. In this paper, we propose a projection-type dualview 3D display based on integral imaging. To separate the images clearly, a high-resolution projection device and lenticular lenses are used. The 3D image in each direction is integrated by a lens array that allows high brightness. In the experiments, we implement the proposed system and observe the two different 3D images in each viewing zone. 2. Principle A.
Concept of the Proposed System
Figure 1 shows the schematic diagram of the proposed system. The projector generates elemental images for the dual-view 3D images. The elemental
Fig. 1. Schematic diagram of the proposed dual-view 3D display system.
images from the projector are parallelized by the collimating lens. When the collimated elemental images are projected on the lenticular lenses, the propagation directions of the projected elemental images are divided into left and right. The lens array placed in front of the lenticular lenses generates a left and right viewing zone from the separated elemental images, and different 3D images are observed in each viewing zone with sharing the lens array. In this configuration, we implement the dualview 3D display based on integral imaging using lenticular lenses and a lens array. A projection-type display is used for the proposed system, because the projection type has significant advantages in collimating the images, aligning them to the lenticular lenses, and implementing high resolution in a small screen [14]. B. Dual-View Generation Using Lenticular Lens
The most important concept of a dual-view display system is how to generate two different views. In the proposed system, lenticular lenses are used for producing a dual-view property. The lenticular lenses are an array of cylindrical concave lenses. Since the normal incidence ray into the lenticular lens passes the focal point, the direction of the ray after passing through the lenticular lens can be adjusted by changing the location of the incidence. These characteristics enable division of the collimated image into different directions. We can calculate the separation angle θsep between the left viewing zone and right viewing zone by using a geometrical relation, as shown in Fig. 2. It is expressed by θsep 2 tan−1
D ; fl
(1)
where D is the distance of the ray from the center of each lenticular lens, which is the location of the incidence, and f l is the focal length of the lenticular lens. Unlike the previous dual-view integral imaging systems [11,12], by changing D and f l , the separation 20 September 2014 / Vol. 53, No. 27 / APPLIED OPTICS
G13
Fig. 2. Splitting elemental images at lenticular lenses.
angle of the proposed system can be adjusted flexibly according to the application, while that of the previous systems is fixed. To divide the whole image from the projector into two different elemental images for left and right viewing zones, the two elemental images should be interwoven to propagate into proper viewing zones. The mapping position of each elemental image is determined by θsep from Eq. (1). The elemental images of the left and right viewing zones are interwoven alternatively line by line, as shown in Fig. 2, while staying a distance D from the center of the lenticular lens. In this case, the maximum number of pixels that one lenticular lens can contain is two, which is the same as the number of the views. If there are more than two pixels in the lenticular lens, they generate more than two views with different separation angles. The extra view causes crosstalk between the viewing zones and makes the 3D images blurred. Instead of increasing the number of pixels in the lenticular lens, if the lenticules per inch (LPI) of the lenticular lenses is higher, the visibility of each view will be improved. C. Elemental Image Generation for Off-Axis Integral Imaging System
In an on-axis integral imaging system, the normal axis of the elemental image surface and the optical axis of the elemental lens are on the same axis. Therefore, when the elemental images are generated, 3D objects are picked up with an on-axis direction, like in Fig. 3(a). The elemental lenses act like cameras, and they capture the disparities of the 3D objects in different directions as elemental images. In Fig. 3(b), the direction of the rays in the dualview system and the surface normal direction of the lens array have an off-axis configuration. The elemental images for the left and right viewing zones should be generated with consideration of the off-axis G14
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
Fig. 3. Elemental image generation method for (a) on-axis integral imaging and (b) off-axis integral imaging.
configuration. The elemental lens of the system acts as a camera, as in conventional on-axis integral imaging. However, the axis that connects the center of the elemental lens and the elemental image is slanted by θsep ∕2, as shown in Fig. 3(b). The lens array captures disparities of the 3D objects obliquely, which means that the elemental images in the off-axis configuration can be generated by shifting the lens array along the horizontal direction. The shifted distance of the elemental images d can be calculated by d g tan
θsep ; 2
(2)
where g is the gap between the lens array and the elemental images. After the elemental images of each viewing zone are generated in the off-axis configuration, the elemental images are interwoven properly to satisfy the dual-view condition as mentioned earlier. Figures 4(a) and 4(b) represent the elemental images
Fig. 5. Viewing angle of (a) on-axis and (b) off-axis integral imaging.
inclined by θsep ∕2. Assuming that all of the elemental lenses in the lens array are used for reconstructing 3D images, the minimum distance between the lens array and the viewing zones L is defined by L m − 1g;
Fig. 4. Computer-generated elemental images for (a) left viewing zone, (b) right viewing zone, and (c) interwoven elemental images.
(5)
where m is the number of the elemental lenses on one side of the lens array. Inside each viewing zone, the left and right elemental images are integrated by sharing the lens array. Since the viewing zones are separated completely without any overlapping zone, the proposed system can display two different 3D images independently without crosstalk. 3. Simulations
of the left and right viewing zones, and Fig. 4(c) represents the interwoven elemental images for the dual-view 3D display system. D.
Viewing Parameters Analysis of the Proposed System
The on-axis integral imaging system reconstructs 3D images on the normal axis, and the viewing angle θvo is simply calculated as shown in Fig. 5(a). It is expressed by tan
θvo p ; 2 2g
The visibility of 3D images in the proposed system is mainly affected by both the LPI of the lenticular lenses and the numerical aperture (NA) of the lens array. To confirm the principles of the proposed system and to evaluate the visibility of the reconstructed 3D images in each viewing zone, we perform optical simulations of the proposed system by using LightTools 8.1, which is an optical simulation tool based on the nonsequential Monte Carlo ray-tracing
(3)
where p is the pitch of the elemental lens. In the proposed system, the separated elemental images are reconstructed on the axis inclined by θsep ∕2, as shown in Fig. 5(b). In the off-axis configuration, the viewing angle θv can be calculated by tan θv
1 1 tan2
θsep 2
g p
p − 4g
:
(4)
Figure 6 shows the schematic diagram of the left and right viewing zones in the proposed system. The viewing zones are constructed along the axes
Fig. 6. Viewing parameters of the proposed system. 20 September 2014 / Vol. 53, No. 27 / APPLIED OPTICS
G15
Fig. 7. Schematic diagram of the optical arrangement in LightTools simulation.
technique. In the simulations, the lenticular lenses separate the propagation directions of the interwoven image into two different directions, and the lens array integrates the elemental images into a 3D image. Two virtual cameras are placed in the left and right viewing zones to observe the reconstructed images in each viewing zone. The resolution of the elemental images used in the simulations is 1920 × 1080 pixels, and their pixel pitch is 0.042 mm. The schematic diagram of the optical arrangements in the simulations is shown in Fig. 7, and the detailed specifications are listed in Table 1. The visibility of the separated elemental images is proportional to the LPI of the lenticular lens. Since the number of lenticular lenses for each elemental lens of the lens array determines the resolution of the elemental images for each viewing zone, the high LPI of lenticular lenses can improve the visibility of the reconstructed 3D images by providing highresolution elemental images. Moreover, since the proposed system is based on off-axis integral imaging, some of the obliquely incident rays cannot pass through the lens array and cannot be integrated into the 3D images. To evaluate these relations, the simulations are performed with varying LPI of the lenticular lenses and the focal length of the lens array, as shown in Fig. 8. In the simulation, the elemental images are generated to present a number “3” at the left viewing zone and a letter “D” at the right viewing zone. The gap between the elemental images and the lens array is adjusted to reconstruct the 3D images at 50 mm in front of the lens array. As shown in Fig. 8(a), the different symbols “3” and “D” can be observed from different viewing positions Table 1.
Specifications of the Simulations
Simulations
1
2
3
Lenticular lenses
LPI Thickness (mm) Focal length (mm)
50 0.61 0.61
100 0.61 0.61
100 0.61 0.61
Lens array
Lens pitch (mm) 5 5 5 Number of lenses 15 × 15 15 × 15 15 × 15 Focal length (mm) 10 10 25
G16
that satisfy the dual-view property of the proposed method. In the reconstructed images, periodic black lines appear between the color information of the 3D images. Since the elemental images are interwoven to generate the dual-view characteristic, the thin black lines come from the separation distance between the pixels that is intended to separate the elemental images into two different directions. The thick black lines appear when some of the rays cannot pass through the elemental lens of the lens array because the NA of the lens array limits the amount of rays. When the rays are obliquely incident on the lens, the limitation by the NA is more severe than the normal incidence. In this simulation, the visibility of the reconstructed 3D images can be evaluated by calculating the maximum number of the color lines in the horizontal direction, and we define the number as a visibility factor. The deteriorated visibility of the reconstructed images, which is caused by the thin black lines, can be improved by using high LPI lenticular lenses and densely interwoven elemental images. By comparing Figs. 8(a) and 8(b), the visibility factors of the 3D images are improved in the simulations by increasing LPI of the lenticular lenses from 50 to 100. In the left viewing zone, the visibility factor of the 3D image is improved from 10 to 14, and in the right viewing zone it is improved from 14 to 20. Since the thick black lines are caused by the limited NA of the lens array, they can be removed by using a
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
Fig. 8. Reconstructed 3D images in the simulation by using lenticular lenses of (a) 50 LPI, (b) 100 LPI, and (c) 100 LPI. The lens arrays in the simulation have focal lengths of (a) 10 mm, (b) 10 mm, and (c) 20 mm.
high NA lens array. The NA of the lens array in the simulation is adjusted by varying the focal length of the lens array while maintaining its pitch. By comparing Figs. 8(b) and 8(c), the visibility factor of the reconstructed 3D images is improved in the simulation by increasing the NA of the lens array. In the left viewing zone, the visibility factor of the 3D images is improved from 14 to 29, and in the right viewing zone, it is improved from 20 to 43. According to the simulation results, the visibility of the dualview 3D images will be improved by using the high LPI lenticular lenses and the high NA lens array. 4. Experiments
To investigate the feasibility of the proposed system, we implement the experimental setup as shown in Fig. 9. For the projection part, we use a Panasonic PT-AE1000 projector and a telecentric lens of Edmund Optics. The elemental images have the resolution of 1920 × 1080, and they are collimated by the telecentric lens. The lenticular lenses give the directivities to the collimated images, and a dual view is generated. We use lenticular lenses of 50 LPI, which means about two lenticules per millimeter. The lens array with 5 mm lens pitch and 10 mm focal length is used. An elemental lens of the lens array contains about 10 lenticules. The pixels of the interwoven elemental images are mapped with uniform distance to fit in the size of the lenticular lens. The separation angle θsep is determined by that distance, according to Eq. (1). In a projection-type imaging system, the exit pupil of the projection optics limits the visible image area and lowers the fill factor when we directly observe the projected image without diffuser. To observe the entire image from the projection-type imaging system, a diffusive screen is required in practice. In the proposed system, a vertical diffuser (PC6001-80, manufactured by Fusion Optix) is placed in front of the lenticular lenses for preserving the directivity of the dual-view system and solving the low fill factor problem due to the exit pupil of the projector. The separated elemental images are integrated by the lens array, and 3D images are reconstructed in each viewing zone. The left viewing zone is formed from −26° to −5° in front of the lens array, and the right viewing zone is formed from 5° to 26° in front of the lens array, according to Eqs. (1) and (4). The detailed specifications of the prototype are presented in Table 2.
Fig. 9. Experimental setup of the proposed system.
Table 2.
Specifications of the Prototype
Elemental images
Resolution (pixels) Pixel pitch (mm)
1920 × 1080 0.042
Lenticular lenses
LPI Thickness (mm) Focal length (mm)
50 0.61 0.61
Vertical diffuser
Diffusing angle (°) Horizontal Vertical
60 1
Lens array
Lens pitch (mm) Number of lenses Focal length (mm)
5×5 15 × 15 10
Viewing parameters
θsep (°) θv (°) g (mm) L (mm)
32 21 12.5 175
To verify that the 3D images of each viewing zone are reconstructed correctly, we use 3D images with four letters placed in different depths. In the left viewing zone, a letter “A” is integrated at 60 mm in front of the lens array and a letter “B” is at 40 mm. In the right viewing zone, a letter “C” is integrated at 60 mm in front of the lens array and a letter “D” is at 40 mm. As shown in Fig. 10, the two different 3D images are reconstructed at each
Fig. 10. Reconstructed 3D images of (a) left viewing zone and (b) right viewing zone. 20 September 2014 / Vol. 53, No. 27 / APPLIED OPTICS
G17
viewing zone. Figure 10(a) presents the reconstructed 3D image of the left viewing zone, and Fig. 10(b) presents the reconstructed 3D images of the right viewing zone. We can find that the relative positions of the two letters are changing with viewing directions in each viewing zone, and the disparities of both vertical and horizontal directions are verified. Since the 3D images are reconstructed through the off-axis configuration, there are distortions of the images by aberrations. It can be improved by applying the predistortion method: the image distortions caused by off-axis configuration can be estimated and compensated during pre-image processing [15].
2.
3.
4.
5.
5. Conclusion
In this paper, we proposed a projection-type dualview 3D display system using integral imaging that provides two completely separated viewing zones and full-parallax 3D images in each viewing zone. The projection image is collimated by the telecentric lens, and the lenticular lenses split the image into two different directions. The lens array generates 3D images in the left and right viewing zones. The characteristics and parameters of the proposed system are analyzed by using the optical simulation. Through the experiment, we confirm that the viewing zones are properly separated, and the perspectives of 3D images from computer-generated elemental images are observed in each viewing zone. The visibility of the 3D images can be improved by using high LPI lenticular lenses and a high NA lens array. We expect that the proposed system can be applied in various areas that have potential demand for the dual-view 3D display system, such as the automotive industry or medical field.
6.
7.
8.
9.
10.
11.
12.
This research was supported by “The CrossMinistry Giga KOREA Project” of The Ministry of Science, ICT and Future Planning, Korea. [GK13D0200, Development of Super Multi-View (SMV) Display Providing Real-Time Interaction].
13.
References
15.
1. J. Hong, Y. Kim, H.-J. Choi, J. Hahn, J.-H. Park, H. Kim, S.-W. Min, N. Chen, and B. Lee, “Three-dimensional display
G18
APPLIED OPTICS / Vol. 53, No. 27 / 20 September 2014
14.
technologies of recent interest: principles, status, and issues,” Appl. Opt. 50, H87–H115 (2011). S. S. Kim, B. H. You, H. Choi, B. H. Berkeley, and N. D. Kim, “World’s first 240 Hz TFT-LCD technology for full-HD LCD-TV and its application to 3D display,” SID Int. Symp. Dig. Tech. Pap. 40, 424–427 (2009). D.-S. Kim, S.-M. Park, J.-H. Jung, and D.-C. Hwang, “New 240 Hz driving method for full HD and high quality 3D LCD TV,” SID Int. Symp. Dig. Tech. Pap. 41, 762–765 (2010). H. Kang, S.-D. Roh, I.-S. Baik, H.-J. Jung, W.-N. Jeong, J.-K. Shin, and I.-J. Chung, “A novel polarizer glasses-type 3D displays with a patterned retarder,” SID Int. Symp. Dig. Tech. Pap. 41, 1–4 (2010). J.-Y. Son, V. V. Saveljev, Y.-J. Choi, J.-E. Bahn, H.-H. Choi, and S.-K. Kim, “Parameters for designing autostereoscopic imaging systems based on lenticular, parallax barrier and integral photography plates,” Opt. Eng. 42, 3326–3333 (2003). J.-Y. Son, V. Saveljev, J.-S. Kim, S.-S. Kim, and B. Javidi, “Viewing zones in three-dimensional imaging systems based on lenticular, parallax-barrier, and microlens-array plates,” Appl. Opt. 43, 4985–4992 (2004). J.-H. Park, K. Hong, and B. Lee, “Recent progress in threedimensional information processing based on integral imaging,” Appl. Opt. 48, H77–H94 (2009). L.-Y. Chen, Y.-Y. Chen, H.-L. Hou, Y. Mo, and C.-Y. Chiu, “Development of the world-wide largest 110′ 4K2K 3D TFTLCD display,” SID Int. Symp. Dig. Tech. Pap. 44, 100–103 (2013). S.-G. Park, J. Yeom, Y. Jeong, N. Chen, J.-Y. Hong, and B. Lee, “Recent issues on integral imaging and its applications,” J. Inf. Disp. 15, 37–46 (2014). H. Choi, S.-W. Min, S. Jung, J.-H. Park, and B. Lee, “Multipleviewing-zone integral imaging using a dynamic barrier array for three-dimensional displays,” Opt. Express 11, 927–932 (2003). F. Wu, H. Deng, C.-G. Luo, D.-H. Li, and Q.-H. Wang, “Dualview integral imaging three-dimensional display,” Appl. Opt. 52, 4911–4914 (2013). F. Wu, Q.-H. Wang, C.-G. Luo, D.-H. Li, and H. Deng, “Dualview integral imaging 3D display using polarizer parallax barriers,” Appl. Opt. 53, 2037–2039 (2014). Y. Kim, J. Kim, K. Hong, H. K. Yang, J.-H. Jung, H. Choi, S.-W. Min, J.-M. Seo, J.-M. Hwang, and B. Lee, “Accommodative response of integral imaging in near distance,” J. Disp. Technol. 8, 70–78 (2012). Y. Kim, K. Hong, J. Yeom, J. Hong, J.-H. Jung, Y. W. Lee, J.-H. Park, and B. Lee, “A frontal projection-type three-dimensional display,” Opt. Express 20, 20130–20138 (2012). Y. M. Kim, K.-M. Jung, and S.-W. Min, “Analysis of off-axis integral floating system using concave mirror,” J. Opt. Soc. Korea 16, 270–276 (2012).
