Case Report
Urologia
Received: September 18, 2013 Accepted: October 28, 2013 Published online: August 13, 2014
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
Internationalis
Head-Mounted Display for a Personal Integrated Image Monitoring System: Ureteral Stent Placement Soichiro Yoshida a Kazunori Kihara a, b Hideki Takeshita a Yasukazu Nakanishi a Toshiki Kijima a Junichiro Ishioka a Yoh Matsuoka a Noboru Numao a, b Kazutaka Saito a Yasuhisa Fujii a
a
Department of Urology, Tokyo Medical and Dental University Graduate School, and b Center for Minimally Invasive Surgery, Tokyo Medical and Dental University, University Hospital of Medicine, Tokyo, Japan
Key Words Cystoscopy · Endourology · Stent · Ureteral stent
Abstract The personal head-mounted display (HMD) has emerged as a novel image monitoring system. We present here the application of a high-definition organic electroluminescent binocular HMD in ureteral stent placement. Our HMD system displayed multiple forms of information such as integrated, sharp, high-contrast images using a four-split screen or a picture-in-picture technique both seamlessly and synchronously. When both the operator and the assistant wore an HMD, they could continuously and simultaneously monitor the cystoscopic and fluoroscopic images in an ergonomically natural position. Furthermore, each participant was able to modulate the displayed images depending on the procedure. In all five cases, both the operator and the assistant successfully used this system with no unfavorable event. No participants experienced any HMD wear-related adverse effects. We therefore believe this HMD system might be potentially beneficial during ureteral stent placement procedures. Furthermore, it is compact, easily introduced and affordable. © 2014 S. Karger AG, Basel
© 2014 S. Karger AG, Basel 0042–1138/14/0941–0117$39.50/0 E-Mail [email protected] www.karger.com/uin
Introduction
Retrograde ureteral stent insertion is performed with a flexible/rigid cystoscope and a guidewire under fluoroscopic guidance using separate, fixed monitors for each image. During this procedure, it is important to observe the ureteral orifice through the cystoscope and to monitor the position of the ureteral stent under fluoroscopic visualization continuously and simultaneously. In endoscopic surgery, all procedures heretofore have been carried out while observing a flat panel display. This concept has dramatically changed in the last few decades thanks to the introduction of the da Vinci Surgical System (Intuitive Surgical, Sunnyvale, Calif., USA) [1]. This system provides the surgeon with a three-dimensional (3-D) view as he sits at a console with his head tilted forward. At present, however, we do not use this system for minor surgery or outside the operating room due to the space requirements of the bulky control console, the extraordinary expensiveness of the equipment and the running costs. The personal head-mounted display (HMD) system has been evaluated as a novel image monitoring system [2]. The application of an HMD system has been preKazunori Kihara, MD, PhD Department of Urology, Tokyo Medical and Dental University 1-5-45 Yushima, Bunkyo-ku Tokyo 113-8519 (Japan) E-Mail k-kihara.uro @ tmd.ac.jp
liminarily evaluated in ultrasound scanning, anesthesia management and laparoscopic surgery [3–6]. The latest HMDs have seen spectacular improvement in terms of imaging quality, weight and wearing comfort. The HMZT2 (Sony Corporation, Tokyo, Japan) is a 330-g binocular HMD composed of 0.7-inch organic light-emitting diode screens (resolution 1,280 × 720 pixels) for each eye, which provide the wearer with a sharp, high-contrast image. We previously combined the HMZ-T2 with a high-definition 3-D endoscope to create the TMDU-S-3D system. We then used this system during a minimally invasive surgical procedure using gasless single-port radical nephrectomy techniques [7–10]. The TMDU-S-3D system presents the surgeon with higher-quality magnified 3-D images in front of the eyes regardless of head position, and simultaneously allows direct vision when moving the angle of sight downward. The significant utility of the TMDU-S-3D system motivated us to use the HMD system for a different urological surgical procedure. We therefore developed a new personal HMD system which makes it possible to integrate multiple forms of information in front of the eyes. We present here an introduction of the HMD system for the retrograde insertion of a ureteral stent.
Materials and Methods Informed consent was obtained following complete description of the retrograde insertion of a ureteral stent procedure using an HMD system, and the procedure was carried out with the approval of our university’s ethics committee. Three surgeons, who were familiar with gasless single-port surgery using the TMDU-S3D system [7], performed the procedures in 5 cases. Patient characteristics are shown in table 1. Ureteral stent insertion was performed using a two-dimensional (2-D) rigid cystoscope (Karl Storz, Culver City, Calif., USA) and guidewire under fluoroscopic guidance using a retrograde approach. The fluoroscopic image (Siemens, Erlangen, Germany), the cystoscopic image, the patient’s vital signs and the view from a head-mounted camera attached on the HMD were connected to a high-definition HMD (HMZ-T2) through a multiplexer (VPM-H1, MEDIA Corporation, Hyogo, Japan). These images were integrated in the HMD using a four-split screen or a picture-in-picture image technique (fig. 1a). Both the operator and the assistant wore an HMD during all parts of the procedure (fig. 1b). These integrated images could be displayed both seamlessly and synchronously. Under lumbar or general anesthesia, the patient was placed in the lithotomy position. The guidewire and access ureteral catheter were inserted into the ureteral meatus through the working channel of the rigid cystoscope and then advanced to the renal pelvis. Next, the 6-Fr ureteral stent (Percuflex Plus, Boston Scientific Corporation, Natick, Mass., USA) was advanced over the
118
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
Table 1. Patient characteristics and purpose of stent placement
Case No. Age
Sex
Purpose of stent placement
1 2 3 4 5
male female female male male
to relieve ureteral obstruction as an adjunct to ureteroscopy as an adjunct to ureteroscopy as an adjunct to partial nephrectomy as an adjunct to partial nephrectomy
57 61 45 60 54
guidewire. During this stage, the ureteral meatus was visually observed through the cystoscope and the stent position was fluoroscopically monitored in order to avoid either migration of the lower end of the stent into the ureter or the guidewire becoming slack in the bladder.
Results
By wearing an HMD system, all the participants could simultaneously and continuously monitor the cystoscopic and fluoroscopic view in an ergonomically natural position during the catheter and cystoscopic stages. Furthermore, based on the procedure, each HMD wearer could arrange the images displayed in the HMD (fig. 2). The HMD wearer could use either the view from a headmounted camera attached on the HMD or direct vision by glancing downward (fig. 1). None of the operators experienced any HMD wear-related adverse effects or claimed any discomfort. In all cases, ureteral stent placement was successfully performed with no remarkable unfavorable events, and operating times were within the usual limit.
Discussion
In the current study, we applied the HMD system to the insertion of a ureteral stent. During this procedure, the HMD wearer was able to continuously and simultaneously assess multiple forms of information displayed in the HMD. Moreover, this HMD system is compact, easily introduced and affordable. It may offer potential advantages not only for urological surgical procedures but also for other medical applications, including esophagogastroduodenoscopy and bronchoscopy. Although the efficacy of the HMD system in medical practice needs to be carefully evaluated, its potential usefulness is quite high, and it may help refine radiological and endoscopic image monitoring. Yoshida et al.
Color version available online
a
Fig. 1. a Captured image of displayed data
b Color version available online
including the cystoscopic and fluoroscopic image, the patient’s vital signs and the view from a head-mounted camera attached to the HMD. These multiple images can be displayed simultaneously and seamlessly. Additionally, all participants can simultaneously share the integrated image. Furthermore, based on the procedure, each HMD wearer can arrange the images displayed in the HMD using a four-split screen or a picture-in-picture technique. b Scene of ureteral stent placement using the HMD. Both the operator and the assistant wear an HMD during all stages of the procedure.
a
b
Fig. 2. Captured images of the displayed data at the scene of retrograde pyelography. The cystoscopic image was mainly displayed in the HMD worn by the operator (a), while the fluoroscopic image was mainly displayed in the HMD of the assistant (b).
Ureteral Stent Placement Using a Head-Mounted Display
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
119
Our concept proposed in the current paper is novel in terms of using the HMD as a personal integrated image monitoring system. In this system, all participants were able to wear an HMD and could view shared but personalized images. The participants could modulate the displayed images depending on the stage of the procedure (e.g., selecting the type or size of the displayed image or changing the arrangement of the combined image). We tested the retrograde insertion of a ureteral stent using an HMD system combined with a 2-D endoscope because, unfortunately, a 3-D endoscope is not commercially
available for cystoscopy. Fortunately, it is easy to connect a range of different devices to the HMD system. Therefore, once a 3-D endoscope or higher-quality see-through HMD becomes available, we can easily introduce it to this HMD system and customize the navigation system. Disclosure Statement Dr. Kihara has received research funding from Sony Corporation (Tokyo, Japan), but the sponsor had no control over the interpretation, writing or publication of this work.
References 1 Binder J, Bräutigam R, Jonas D, Bentas W: Robotic surgery in urology: fact or fantasy? BJU Int 2004;94:1183–1187. 2 Rolland JP, Fuchs H: Optical versus video seethrough head-mounted displays in medical visualization. Presence 2000;9:287–309. 3 Havukumpu J, Takatalo J, Nyman G, Hakkinen J: Head-mounted displays in ultrasound scanning; in Asai K (ed): Human Computer Interaction: New Developments. Rijeka, Intech, 2008, pp 31–40. 4 Liu D, Jenkins SA, Sanderson PM, Fabian P, Russell WJ: Monitoring with head-mounted displays in general anesthesia: a clinical evaluation in the operating room. Anesth Analg 2010;110:1032–1038.
120
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
5 Liu D, Jenkins SA, Sanderson PM, Watson MO, Leane T, Kruys A, Russell WJ: Monitoring with head-mounted displays: performance and safety in a full-scale simulator and part-task trainer. Anesth Analg 2009;109:1135–1146. 6 Prendergast CJ, Ryder BA, Abodeely A, Muratore CS, Crawford GP, Luks FI: Surgical performance with head-mounted displays in laparoscopic surgery. J Laparoendosc Adv Surg Tech A 2009;19(suppl 1):S237–S240. 7 Kihara K, Fujii Y, Masuda H, Saito K, Koga F, Matsuoka Y, Numao N, Kojima K: New threedimensional head-mounted display system, TMDU-S-3D system, for minimally invasive surgery application: procedures for gasless single-port radical nephrectomy. Int J Urol 2012;19:886–889.
8 Kihara K, Kageyama Y, Yano M, Kobayashi T, Kawakami S, Fujii Y, Masuda H, Hyochi N: Portless endoscopic radical nephrectomy via a single minimum incision in 80 patients. Int J Urol 2004;11:714–720. 9 Kihara K, Kawakami S, Fujii Y, Masuda H, Koga F, Saito K: Gasless single port access radical nephrectomy. Eur Urol Suppl 2009;8:392. 10 Kihara K, Kawakami S, Fujii Y, Masuda H, Koga F: Gasless single-port access endoscopic surgery in urology: minimum incision endoscopic surgery, MIES. Int J Urol 2009;16:791– 800.
Yoshida et al.
Copyright: S. Karger AG, Basel 2015. Reproduced with the permission of S. Karger AG, Basel. Further reproduction or distribution (electronic or otherwise) is prohibited without permission from the copyright holder.
Urologia
Received: September 18, 2013 Accepted: October 28, 2013 Published online: August 13, 2014
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
Internationalis
Head-Mounted Display for a Personal Integrated Image Monitoring System: Ureteral Stent Placement Soichiro Yoshida a Kazunori Kihara a, b Hideki Takeshita a Yasukazu Nakanishi a Toshiki Kijima a Junichiro Ishioka a Yoh Matsuoka a Noboru Numao a, b Kazutaka Saito a Yasuhisa Fujii a
a
Department of Urology, Tokyo Medical and Dental University Graduate School, and b Center for Minimally Invasive Surgery, Tokyo Medical and Dental University, University Hospital of Medicine, Tokyo, Japan
Key Words Cystoscopy · Endourology · Stent · Ureteral stent
Abstract The personal head-mounted display (HMD) has emerged as a novel image monitoring system. We present here the application of a high-definition organic electroluminescent binocular HMD in ureteral stent placement. Our HMD system displayed multiple forms of information such as integrated, sharp, high-contrast images using a four-split screen or a picture-in-picture technique both seamlessly and synchronously. When both the operator and the assistant wore an HMD, they could continuously and simultaneously monitor the cystoscopic and fluoroscopic images in an ergonomically natural position. Furthermore, each participant was able to modulate the displayed images depending on the procedure. In all five cases, both the operator and the assistant successfully used this system with no unfavorable event. No participants experienced any HMD wear-related adverse effects. We therefore believe this HMD system might be potentially beneficial during ureteral stent placement procedures. Furthermore, it is compact, easily introduced and affordable. © 2014 S. Karger AG, Basel
© 2014 S. Karger AG, Basel 0042–1138/14/0941–0117$39.50/0 E-Mail [email protected] www.karger.com/uin
Introduction
Retrograde ureteral stent insertion is performed with a flexible/rigid cystoscope and a guidewire under fluoroscopic guidance using separate, fixed monitors for each image. During this procedure, it is important to observe the ureteral orifice through the cystoscope and to monitor the position of the ureteral stent under fluoroscopic visualization continuously and simultaneously. In endoscopic surgery, all procedures heretofore have been carried out while observing a flat panel display. This concept has dramatically changed in the last few decades thanks to the introduction of the da Vinci Surgical System (Intuitive Surgical, Sunnyvale, Calif., USA) [1]. This system provides the surgeon with a three-dimensional (3-D) view as he sits at a console with his head tilted forward. At present, however, we do not use this system for minor surgery or outside the operating room due to the space requirements of the bulky control console, the extraordinary expensiveness of the equipment and the running costs. The personal head-mounted display (HMD) system has been evaluated as a novel image monitoring system [2]. The application of an HMD system has been preKazunori Kihara, MD, PhD Department of Urology, Tokyo Medical and Dental University 1-5-45 Yushima, Bunkyo-ku Tokyo 113-8519 (Japan) E-Mail k-kihara.uro @ tmd.ac.jp
liminarily evaluated in ultrasound scanning, anesthesia management and laparoscopic surgery [3–6]. The latest HMDs have seen spectacular improvement in terms of imaging quality, weight and wearing comfort. The HMZT2 (Sony Corporation, Tokyo, Japan) is a 330-g binocular HMD composed of 0.7-inch organic light-emitting diode screens (resolution 1,280 × 720 pixels) for each eye, which provide the wearer with a sharp, high-contrast image. We previously combined the HMZ-T2 with a high-definition 3-D endoscope to create the TMDU-S-3D system. We then used this system during a minimally invasive surgical procedure using gasless single-port radical nephrectomy techniques [7–10]. The TMDU-S-3D system presents the surgeon with higher-quality magnified 3-D images in front of the eyes regardless of head position, and simultaneously allows direct vision when moving the angle of sight downward. The significant utility of the TMDU-S-3D system motivated us to use the HMD system for a different urological surgical procedure. We therefore developed a new personal HMD system which makes it possible to integrate multiple forms of information in front of the eyes. We present here an introduction of the HMD system for the retrograde insertion of a ureteral stent.
Materials and Methods Informed consent was obtained following complete description of the retrograde insertion of a ureteral stent procedure using an HMD system, and the procedure was carried out with the approval of our university’s ethics committee. Three surgeons, who were familiar with gasless single-port surgery using the TMDU-S3D system [7], performed the procedures in 5 cases. Patient characteristics are shown in table 1. Ureteral stent insertion was performed using a two-dimensional (2-D) rigid cystoscope (Karl Storz, Culver City, Calif., USA) and guidewire under fluoroscopic guidance using a retrograde approach. The fluoroscopic image (Siemens, Erlangen, Germany), the cystoscopic image, the patient’s vital signs and the view from a head-mounted camera attached on the HMD were connected to a high-definition HMD (HMZ-T2) through a multiplexer (VPM-H1, MEDIA Corporation, Hyogo, Japan). These images were integrated in the HMD using a four-split screen or a picture-in-picture image technique (fig. 1a). Both the operator and the assistant wore an HMD during all parts of the procedure (fig. 1b). These integrated images could be displayed both seamlessly and synchronously. Under lumbar or general anesthesia, the patient was placed in the lithotomy position. The guidewire and access ureteral catheter were inserted into the ureteral meatus through the working channel of the rigid cystoscope and then advanced to the renal pelvis. Next, the 6-Fr ureteral stent (Percuflex Plus, Boston Scientific Corporation, Natick, Mass., USA) was advanced over the
118
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
Table 1. Patient characteristics and purpose of stent placement
Case No. Age
Sex
Purpose of stent placement
1 2 3 4 5
male female female male male
to relieve ureteral obstruction as an adjunct to ureteroscopy as an adjunct to ureteroscopy as an adjunct to partial nephrectomy as an adjunct to partial nephrectomy
57 61 45 60 54
guidewire. During this stage, the ureteral meatus was visually observed through the cystoscope and the stent position was fluoroscopically monitored in order to avoid either migration of the lower end of the stent into the ureter or the guidewire becoming slack in the bladder.
Results
By wearing an HMD system, all the participants could simultaneously and continuously monitor the cystoscopic and fluoroscopic view in an ergonomically natural position during the catheter and cystoscopic stages. Furthermore, based on the procedure, each HMD wearer could arrange the images displayed in the HMD (fig. 2). The HMD wearer could use either the view from a headmounted camera attached on the HMD or direct vision by glancing downward (fig. 1). None of the operators experienced any HMD wear-related adverse effects or claimed any discomfort. In all cases, ureteral stent placement was successfully performed with no remarkable unfavorable events, and operating times were within the usual limit.
Discussion
In the current study, we applied the HMD system to the insertion of a ureteral stent. During this procedure, the HMD wearer was able to continuously and simultaneously assess multiple forms of information displayed in the HMD. Moreover, this HMD system is compact, easily introduced and affordable. It may offer potential advantages not only for urological surgical procedures but also for other medical applications, including esophagogastroduodenoscopy and bronchoscopy. Although the efficacy of the HMD system in medical practice needs to be carefully evaluated, its potential usefulness is quite high, and it may help refine radiological and endoscopic image monitoring. Yoshida et al.
Color version available online
a
Fig. 1. a Captured image of displayed data
b Color version available online
including the cystoscopic and fluoroscopic image, the patient’s vital signs and the view from a head-mounted camera attached to the HMD. These multiple images can be displayed simultaneously and seamlessly. Additionally, all participants can simultaneously share the integrated image. Furthermore, based on the procedure, each HMD wearer can arrange the images displayed in the HMD using a four-split screen or a picture-in-picture technique. b Scene of ureteral stent placement using the HMD. Both the operator and the assistant wear an HMD during all stages of the procedure.
a
b
Fig. 2. Captured images of the displayed data at the scene of retrograde pyelography. The cystoscopic image was mainly displayed in the HMD worn by the operator (a), while the fluoroscopic image was mainly displayed in the HMD of the assistant (b).
Ureteral Stent Placement Using a Head-Mounted Display
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
119
Our concept proposed in the current paper is novel in terms of using the HMD as a personal integrated image monitoring system. In this system, all participants were able to wear an HMD and could view shared but personalized images. The participants could modulate the displayed images depending on the stage of the procedure (e.g., selecting the type or size of the displayed image or changing the arrangement of the combined image). We tested the retrograde insertion of a ureteral stent using an HMD system combined with a 2-D endoscope because, unfortunately, a 3-D endoscope is not commercially
available for cystoscopy. Fortunately, it is easy to connect a range of different devices to the HMD system. Therefore, once a 3-D endoscope or higher-quality see-through HMD becomes available, we can easily introduce it to this HMD system and customize the navigation system. Disclosure Statement Dr. Kihara has received research funding from Sony Corporation (Tokyo, Japan), but the sponsor had no control over the interpretation, writing or publication of this work.
References 1 Binder J, Bräutigam R, Jonas D, Bentas W: Robotic surgery in urology: fact or fantasy? BJU Int 2004;94:1183–1187. 2 Rolland JP, Fuchs H: Optical versus video seethrough head-mounted displays in medical visualization. Presence 2000;9:287–309. 3 Havukumpu J, Takatalo J, Nyman G, Hakkinen J: Head-mounted displays in ultrasound scanning; in Asai K (ed): Human Computer Interaction: New Developments. Rijeka, Intech, 2008, pp 31–40. 4 Liu D, Jenkins SA, Sanderson PM, Fabian P, Russell WJ: Monitoring with head-mounted displays in general anesthesia: a clinical evaluation in the operating room. Anesth Analg 2010;110:1032–1038.
120
Urol Int 2015;94:117–120 DOI: 10.1159/000356987
5 Liu D, Jenkins SA, Sanderson PM, Watson MO, Leane T, Kruys A, Russell WJ: Monitoring with head-mounted displays: performance and safety in a full-scale simulator and part-task trainer. Anesth Analg 2009;109:1135–1146. 6 Prendergast CJ, Ryder BA, Abodeely A, Muratore CS, Crawford GP, Luks FI: Surgical performance with head-mounted displays in laparoscopic surgery. J Laparoendosc Adv Surg Tech A 2009;19(suppl 1):S237–S240. 7 Kihara K, Fujii Y, Masuda H, Saito K, Koga F, Matsuoka Y, Numao N, Kojima K: New threedimensional head-mounted display system, TMDU-S-3D system, for minimally invasive surgery application: procedures for gasless single-port radical nephrectomy. Int J Urol 2012;19:886–889.
8 Kihara K, Kageyama Y, Yano M, Kobayashi T, Kawakami S, Fujii Y, Masuda H, Hyochi N: Portless endoscopic radical nephrectomy via a single minimum incision in 80 patients. Int J Urol 2004;11:714–720. 9 Kihara K, Kawakami S, Fujii Y, Masuda H, Koga F, Saito K: Gasless single port access radical nephrectomy. Eur Urol Suppl 2009;8:392. 10 Kihara K, Kawakami S, Fujii Y, Masuda H, Koga F: Gasless single-port access endoscopic surgery in urology: minimum incision endoscopic surgery, MIES. Int J Urol 2009;16:791– 800.
Yoshida et al.
Copyright: S. Karger AG, Basel 2015. Reproduced with the permission of S. Karger AG, Basel. Further reproduction or distribution (electronic or otherwise) is prohibited without permission from the copyright holder.
