This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

1

Design of Endoscopic Capsule With Multiple Cameras Yingke Gu, Xiang Xie, Guolin Li, Tianjia Sun, Dan Wang, Zheng Yin, Pengfei Zhang, and Zhihua Wang, Senior Member, IEEE

Abstract—In order to reduce the miss rate of the wireless capsule endoscopy, in this paper, we propose a new system of the endoscopic capsule with multiple cameras. A master-slave architecture, including an efficient bus architecture and a four level clock management architecture, is applied for the Multiple Cameras Endoscopic Capsule (MCEC). For covering more area of the gastrointestinal tract wall with low power, multiple cameras with a smart image capture strategy, including movement sensitive control and camera selection, are used in the MCEC. To reduce the data transfer bandwidth and power consumption to prolong the MCEC’s working life, a low complexity image compressor with PSNR 40.7 dB and compression rate 86% is implemented. A chipset is designed and implemented for the MCEC and a six cameras endoscopic capsule prototype is implemented by using the chipset. With the smart image capture strategy, the coverage rate of the MCEC prototype can achieve 98% and its power consumption is only about 7.1 mW. Index Terms—Capsule endoscope, image compression, multiple cameras, smart image capture.

I. INTRODUCTION

W

IRELESS CAPSULE ENDOSCOPY (WCE), for the first time, allows painless optical imaging of the entire small bowel [1]. It is a revolutionary breakthrough in the field of endoscopy. However, the miss rate of the WCE for the small bowel may reach 20% 30% [2]. Limited visual field and low image acquisition rate of the wireless endoscopic capsule, lead to such high miss rate. Furthermore, the random movement of the capsule is another important reason for the high miss rate. Therefore, in large GI cavities, such as stomach and colon, the situation is even worse. To solve this problem, many research groups and companies are exploiting various technology for improving the performance of the wireless endoscopic capsule under the size and power restrictions. One way is to increase more cameras inside

Manuscript received March 07, 2014; revised June 29, 2014; accepted August 30, 2014. This paper was recommended by Associate Editor A. Bermak. Y. Gu, X. Xie, T. Sun, D. Wang, P. Zhang, and Z. Wang are with the Tsinghua National Laboratory for Information Science and Technology, Tsinghua University, Beijing 100084, China, and also with the Institute of Microelectronics, Tsinghua University, Beijing 100084, China (e-mail: [email protected]). G. Li and Z. Yin are with the Tsinghua National Laboratory for Information Science and Technology, Tsinghua University, Beijing 100084, China, and also with the Department of Electronic Engineering, Tsinghua University, Beijing 100084, China. Color versions of one or more of the figures in this paper are available online at http://ieeexplore.ieee.org. Digital Object Identifier 10.1109/TBCAS.2014.2359012

the capsule for a larger visual field. For example, some wireless capsules equipped with two cameras are using for examining the esophagus and colon [3]. However, its visual field for large GI cavities is still limited and the negative effect of the random movement of the capsule still exists. Hence, the miss rate of this kind of capsule for examining the colon is still high [4]. The coin batteries carried in the wireless capsule cannot support the high power consumption when the capsule maintaining a high image acquisition rate [2]. So, the current available capsule with high image acquisition rate can work for a few minutes and only be used for examining the esophagus [3]. For reducing power consumption, [5] proposed a method utilizing the acceleration information to increase the image acquisition rate only while the rapid movement of the capsule happens. However, this method just improves the image acquisition rate, and cannot solve the problem of limited visual field. Another way is to provide a swallow-able capsule with active locomotion to overcome the negative effect resulted from the random movement of the capsule. Capsules endowed with active locomotion allow direct remote control of the device toward suspicious areas. This method employs two main strategies. The first is to pursue the miniaturization of locomotion systems that are integrated inside the capsule [6]–[8]. The second is to use an external approach where the actuation, generally based on magnetic fields, is outside the capsule [9]–[12]. In this situation, the patient cannot move freely. However, in practice the capsule has power and size restrictions, and human safety must be assured. Hence, the two above-mentioned strategies seem significantly complex and difficult to be realized. In this paper, we propose a new design of Multiple Cameras Endoscopic Capsule (MCEC) with smart control to reduce the miss rate. Multiple cameras are employed to get a larger visual field. Instead of the wireless transmitting, the image data are stored into the flash memory to get a high image acquisition rate. A smart image capture control strategy based upon the movement and attitude information reduces the negative effect of the MCEC’s random movement in the digestive tract. The realization of the MCEC has several design challenges, including the organization of all the modules, low power consumption and miniaturization. In order to meet such challenges, some key techniques are proposed for the multiple cameras system. A master-slave architecture is applied for the MCEC. For lowering power consumption, an efficient bus architecture and a four level clock management architecture are designed. After the MCEC is evacuated, the image data stored inside it need to be read out for the diagnosis. However, the battery inside the capsule is nearly exhausted. In order to enable the

1932-4545 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 2

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

doctor to avoid complicated operations, such as disassembly and connecting wires, a data reader with wireless power transmission ability and wireless data communication ability is proposed. The flash memory can solve the data bandwidth limitation for increasing the image acquisition rate. However, the power consumption of the flash memory is still very high when the image acquisition rate is increased. The image compression technique can effectively solve this problem. The lossless (near-lossless) compression methods applied in the endoscopic capsule [13], [14] always have very low computation complexity and can obtain very high quality reconstructed images (peak signal to noise ratio (PSNR) higher than 45 dB), but their compression rates are low (less than 80%). For improving the compression rate, some lossy compression algorithms have been applied in the endoscopic capsule [15]. This kind of compression method can get high compression rate (more than 90%). However, it increases computation complexity and memory space. Considering the tradeoff between the compression performance (PSNR 40.7 dB and compression rate 86%) and hardware cost, an image compressor based on modified JPEG image compression algorithm has been implemented in this paper. Based on the design techniques above, several kinds of MCEC can be implemented, such as the capsule with two cameras on its both ends. Furthermore, some kinds of spherical MCEC, which have wider field of view, also can be implemented. For example, a six cameras endoscopic capsule prototype named ‘Micro-Ball’ has been implemented. The rest of this paper is organized as follows: Section II presents the multiple cameras endoscopy system architecture design. Section III describes the smart image capture strategy. Section IV discusses the image compression and storage. Section V presents the demonstration system implementation and experimental results. Section VI presents the conclusion. II. MULTIPLE CAMERAS ENDOSCOPY SYSTEM ARCHITECTURE DESIGN A. Master-Slave Architecture in MCEC Increasing the number of cameras could enlarge the MCEC’s vision field. However, integrating all the modules together with a tiny size and without significantly power increasing, has become a difficult problem. The MCEC employs a master-slave architecture to realize the system integration. As shown in Fig. 1, it has one master device and several identical slave devices. The master device is the core part, mainly including the master controller, power supply and global clock management. The slave device is designed for image acquisition tasks, including image capturing, image compression and image storage. From the view point of system integration, the proposed master-slave architecture can make layout and route inside the MCEC easier, and the design of the whole system simplified, structured and flexible. The main modules of the MCEC include master chip, slave chips, image sensors, flash memory chips, wireless power receiving chip, batteries, LEDs, attitude and movement sensing module, etc. For the miniaturization and power reduction, the master chip, slave chip and wireless power receiving chip are

Fig. 1. System level diagram of MCEC.

designed and implemented by ASIC techniques. The master chip controls the whole MCEC and accomplishes the wireless communication function. The slave chip controls the slave device of the MCEC, and contains an image compressor. The wireless power receiving chip can receive wireless energy or battery energy, and provide stable DC power to the MCEC. The rest modules consist of the off-the-shelf items. There is a tradeoff between the performance and the hardware cost for choosing the commercial modules. For example, compared with CCD image sensor, CMOS image sensor is more suitable because of its lower power consumption and acceptable image quality for endoscopy. Especially, OV7660 has been selected because of its lowest power consumption among the commercially available CMOS VGA image sensors. Nand-flash memory is selected because of its high storage density. SAMSUNG K9WAG08U1M is the smallest chip that we can get and has enough memory space. The attitude and movement sensing module contains two MEMS sensors including an accelerator and a magnetometer that have the same selection criteria as above. For the power supply, two 1.55 V@50 mAh coin batteries are used. White LEDs in 0603 SMD package provide illumination when capturing images. All the components used in the MCEC are summarized in Table I. It can be seen that the flash memory chip size limits the miniaturization of the MCEC. In future, a smaller size flash memory chip will have to be got, e.g., bare die, and then a swallow-able MCEC can be implemented. So, it can’t affect the verification of the system design. The common mode of controlling all the modules is shown in Fig. 2(a). It implies that one central controller controls all the modules (only the image sensor is shown for the sake of simplification). However, this mode results in a large amount of

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. GU et al.: DESIGN OF ENDOSCOPIC CAPSULE WITH MULTIPLE CAMERAS

SUMMARIZATION

OF

3

TABLE I COMPONENTS INSIDE

THE

MCEC

different devices, meaning energy efficient. For combining the advantages of the two storage modes, a hybrid storage mode is proposed and shown in Fig. 2(e). Under this hybrid mode, most of the time, each image sensor sends image data to its corresponding flash memory module. This mode also provides the image sensor the ability to send data to other flash memory modules. This ability is useful in case some image sensors’ corresponding flash memory capacity is not sufficient when applying the camera selection strategy. B. Bus Architecture and Multi-Level Clock Management in MCEC

Fig. 2. Architecture of different modes. (a) Central control mode. (b) Hierarchy control mode. (c) Central storage. (d) Distributed storage mode. (e) Hybrid storage mode.

wire connections between the central controller and the modules, to the disadvantage of the miniaturization of the MCEC. Moreover, increasing or decreasing modules under this mode is hard to realize. So, a hierarchy control mode, shown in Fig. 2(b), is proposed to solve such problems. Under this mode, a master controller sends commands to the slave controller and then the slave controller controls its corresponding modules. The data flow inside the MCEC mainly refers to the image data from the image sensor to the flash memory module. Because multiple image sensors are embedded inside the MCEC, the commonly used modes include central storage mode and distributed storage mode. They are shown in Fig. 2(c) and Fig. 2(d). The central storage mode makes all the image sensors to use one flash memory module. The distributed storage mode makes each image sensor to have a corresponding flash memory module. The distributed storage mode can reduce the flash memory chip size on each slave device. Under the distributed storage mode, the data sent into the flash memory module do not need to cross

For organizing all the modules efficiently, an efficient bus architecture has been proposed for the data transfer between the different devices. A four level clock management architecture has also been proposed for reducing the system power consumption. According to the work mode of the MCEC, the data transmission on the bus includes two types: 1) commands from the master device; 2) data from the slave device. Hence, every bus data transfer process can be set starting by the master device, and divided into two phases: Master Command Phase (MCP) and Slave Data Phase (SDP). ‘ ’ shown in Fig. 3, is a functional wire and only driven by the master device. It marks the different phases during the bus data transmission. Considering that the image sensor and flash memory module both have 8-bit width data interface, 8-bit parallel data wires are used for eliminating the process for data conversion. And for reducing the number of wire connections, the data wires are bidirectional and can be driven by master device or any slave device. As shown in Fig. 3, ‘ ’are data wires and ‘DCLK’ is their synchronization clock. When the MCEC is capturing images inside the human’s GI tract, bus data transmission type 1) occupies most part of the bus data transmission. And low power consumption is the chief requirement in this situation. When the MCEC is evacuated, bus data transmission type 2) takes most time of the bus data transmission. And high data rate is the primary requirement. So we need low power consumption in MCP and high data rate in SDP.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 4

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

illustrated in Fig. 5. The first level with the highest priority is the low clock controller, which controls a 24 MHz clock. When the system enters the idle state, the low clock controller will stop the 24 MHz crystal from oscillating. When the low clock controller receives the enable signal, the 24 MHz clock will be enabled and be sent into the required modules. In the second level clock management, with the 24 MHz clock input, the clock splitter provides clocks of different frequencies to different modules. The clock of the idle modules can be disabled by the clock gating controller, and the clock sent to the slave device is buffered because it may cross different PCBs. The first and second level clock management are implemented inside the master device. The third level clock management is applied inside the slave device. It is similar to the second level clock management. Each module in the fourth level has its own clock management. The power simulation shows that this design can save 28% power inside the MCEC compared with a design without clock management.

Fig. 3. Bus architecture.

C. Multiple Cameras Endoscopy System Architecture

Fig. 4. Bus sequence diagram.

The data rate of the bus data transmission is mainly determined by the synchronization clock frequency. The power consumption of the bus data transmission can be indicated as follows:

(1) In (1), is the bus voltage, is the synchronization clock frequency, is the transition probability of one data wire and is the capacitance load of the data wire. According to (1), to meet the requirements including power consumption and data rate, an unsymmetrical transmission mode has been proposed. In MCP, data transmission employs 1-bit serial mode, which means only is used. And the synchronization clock frequency is set as 1 MHz. In SDP, 8-bit parallel mode is adopted and the synchronization clock frequency is set as 12 MHz. The sequence diagram is shown in Fig. 4. The power consumption during MCP only occupies approximately 1% of that during SDP, and the data rate in SDP is about 96 Mb/s. So, this proposed bus architecture not only satisfies the data transmission demands between different devices, but also meets the power consumption and data rate requirements. The MCEC integrates many modules which work at different clock frequencies and at different times. Hence, effective clock management should be designed to decrease the clock frequency of the different modules to be as low as possible for low power dissipation. Moreover, the clocks of the idle modules should be disabled to further reduce the power consumption. A four level clock management architecture is proposed as

The entire multiple cameras endoscopy system architecture shown in Fig. 6 consists of three parts: a swallow-able MCEC, a data reader for reading out the image data stored inside the MCEC after it is evacuated and a workstation used for displaying and post-processing the image. The data reader contains a Wireless Power Transmission Module (WPT Module) having emitting coils to transmit wireless energy to the MCEC. The WPT Module of the data reader and the wireless power receiving chip inside the MCEC can establish a wireless power link between the MCEC and the data reader. After the wireless power link is established, a wireless data link also can be established with the help of the wireless transceiver inside the MCEC and the wireless transceiver of the data reader. Then the data inside the MCEC can be read out and uploaded to the workstation via the PC interface. The compressed image data can be decompressed on the workstation, and the images can be displayed or post-processed for diagnosis. To realize the miniaturization of the MCEC, a master-slave chipset and a power receiving chip are designed and implemented. The master chip architecture is shown in Fig. 7. The control strategy of the MCEC is implemented in the master controller. The first and second level clock management, proposed in above section, are implemented inside the clock manager. The wireless transceiver with its MAC controller receives commands from the data reader and sends the image data to it. The design details of the wireless transceiver can be seen in [16]. The command decoder interprets the commands received and then sends the control signal to the master controller or the slave command encoder. The slave command encoder makes the commands need to be sent to the slave device encoded. Then the master bus controller sends the commands to the slave device. The SPI controller can receive the information for smart image capture control from the Attitude and Movement Sensing Module (AMSM). The smart image capture strategy and the design of the AMSM are described in Section III. The slave chip architecture is shown in Fig. 8. The slave controller receives commands from the slave bus controller and

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. GU et al.: DESIGN OF ENDOSCOPIC CAPSULE WITH MULTIPLE CAMERAS

5

Fig. 5. Four level clock management.

Fig. 6. Multiple cameras endoscopy system architecture.

Fig. 8. Slave chip architecture.

The design details of the image compressor are proposed in Section IV. The LED driver and four LEDs provide illumination when capturing images. The architecture of the wireless power receiving chip is shown in Fig. 9. The chip contains three Rx circuits, which are actually the proposed high efficiency CMOS rectifiers with current Zero-Cross-Point (ZCP) prediction [17]. They work independently and support three receiving coils for omnidirectional receiving. In order to make full use of received energy, the chip also contains a power combination circuit, which is actually the proposed Skipping Booster [17]. It combines the received energy from all directions together and delivers to the low dropout regulator (LDO). This chip also can deliver the energy from coin batteries to the LDO and the choice between wireless power and batteries is decided by the control signal. III. SMART IMAGE CAPTURE Fig. 7. Master chip architecture.

gives control signals to other modules inside the slave chip. The image sensor controller controls the image sensor and receives the image data from the sensor. The flash controller can make the flash memory module to be erased, programmed and read. The image compressor can efficient compress the image data.

In the proposed system, multiple cameras are integrated into the capsule to enlarge the field of view. However, it is not sufficient to reduce the miss rate because of the capsule’s random movement in the GI tract, involving movement at a variable speed and random rotation [18]–[22]. For a lower miss rate with the consideration of low power consumption, a smart image capture strategy utilizing MCEC’s motion and attitude information is proposed. Considering the low power and the small size

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 6

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

Fig. 10. State transition diagram. Fig. 9. Wireless power receiving chip architecture.

limitation, we have applied the attitude and motion sensors including only a magnetometer and an accelerometer for realizing the smart capture strategy. The strategy includes the motion sensitive control of the image frame rate and camera selection with attitude information.

thrust provided by the peristalsis and the weight of the capsule. Comparing such motion parameters with the corresponding parameters of the commercial MEMS accelerometers, it is found that such accelerometers can be used for the capsule motion detection. The detection algorithm is as follows: If

A. Image Frame Rate Control Currently, commercially available capsules have a fixed image acquisition rate in order to be able to capture the entire G.I. tract. However, according to the observation of the WCE image sequence, there are a large amount of redundancies in the WCE image sequence. We can deduce that the constant image acquisition rate is not optimal if the capsule is stationary most of the time. On the other hand, the fixed image acquisition rate may be not high enough, when the capsule moves rapidly. It will lead to the high miss rate of the WCE. To overcome the disadvantages mentioned above, the MCEC changes its image acquisition rate according to its motion state. The motion state is estimated by analyzing the acceleration information. If it is detected no motion, the MCEC enters a monitor mode in which images are captured under a low frame rate. When the MCEC is detected in motion, it returns to a normal mode in which images are captured under a normal rate. When the MCEC is detected in drastic motion, including rapid movement or rotation, it enters a fast mode in which images are captured under a high rate. The three image frame rates in the monitor, normal and fast modes, are set as , and . If the MCEC enters monitor mode, the image frame rate is set to . If it enters normal mode, the image frame rate is set to and lasts image capture periods. An image capture period is defined as times of image capturing, and equals to the number of cameras. If it enters fast mode, the image frame rate is set to and lasts image capture periods. The state transition diagram is shown in Fig. 10. The key step during this control mechanism is the detection of the different states of motion. The motion of the capsule in human GI tract is driven mainly by the gastrointestinal peristalsis. According to the characteristics of gastrointestinal peristalsis [18]–[23], the average peristalsis frequency of the human’s small bowel is about 10 12 times per minute. And the typical acceleration of the capsule is about 1 g 3 g (‘g’ represents gravitational acceleration), which is estimated with the

else if

, motion state detected; , drastic motion state detected;

else, no motion state detected. denotes the th acceleration sample value. represents a moving average filter for noise reduction. and denote two thresholds. The motion sensitive control of the image frame rate reduces redundancy during image capture stage, resulting in power consumption reduction and a more efficient image review process for the physician. B. Camera Selection The current battery discharge characteristics do not allow all the cameras in the MCEC to work simultaneously, as a result, they are set to take turns to capture images. However, the MCEC may rotate during an image capture period, resulting in that the different cameras may capture the same part of the GI tract wall and some area may be missed. Camera selection with the attitude information is applied to overcome the problem caused by the random rotation of the MCEC. The attitude information of the MCEC represents its rotation status. The camera selection strategy can ensure that the cameras selected during an image capture period have different orientations to the GI tract wall. Hence, the captured images can cover as much area of the GI tract wall as possible. Camera selection is a periodical process based on the image capture period. It selects appropriate at time . denotes one of the cameras inside the MCEC. A six cameras endoscopy system named ‘Micro-Ball’ is set as an example for introducing the camera selection strategy. Each camera of the Micro-Ball is placed on the center of a surface of a cube. As shown in Fig. 11, at time , we set the center point of the Micro-Ball as the origin of the coordinates. , and denote X-axis, Y-axis and Z-axis positive direction,

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. GU et al.: DESIGN OF ENDOSCOPIC CAPSULE WITH MULTIPLE CAMERAS

7

Fig. 12. Architecture of AMSM.

(3)

Fig. 11. Coordinate for camera selection.

respectively. where lies. At time . For

denotes the center of the plane

: ,

, the coordinates of are , , , , denotes the side length of the cube

, and shown in Fig. 11. For : , where denotes a Direct-Cosine Matrix (DCM) [24]. Camera selection guarantees that the cameras selected at different times have different orientations. Hence, at time , the camera selection can be defined as

(2) For other kinds of MCEC, we only need to change coordinate system according to the geometrical shape of the MCEC and the layout of the cameras. The rest steps are the same as the description above. The computation of the DCM is the key step in the camera selection algorithm. Considering that the number of cameras to be selected and each camera’s field of view, the DCM computation accuracy requirement is not very high. So we only choose the magnetometer and the accelerometer for the DCM computation. Compared with the common navigation system, the proposed design does not use a gyroscope because of its high power consumption [25], [26]. The DCM is determined by the gravity and geomagnetic field vectors measured at different times. Supposing that the gravity and geomagnetic field vectors measured at are and , respectively, we can deduce (3) ( represents vector product) [24].

As described above, the computation of uses as reference, and is a constant. So the error of is only determined by . According to (3), is computed with the measurements of the sensors at time and . It is unrelated to the measurements at time to . Hence, there is no accumulated error during this process. In order to realize the image frame rate control and camera selection, an Attitude and Movement Sensing Module (AMSM) is proposed. Considering that this module is embedded inside the MCEC, small size and low power consumption are the chief design requirements. Therefore, we choose highly integrated MEMS sensors for the magnetometer and the accelerometer. The architecture of AMSM is shown in Fig. 12. The data from the three-axis accelerometer and three-axis magnetometer are sent to the movement detector and the DCM computer via the sensor interface. The result of the movement detector is used for the image frame rate control and the result of the DCM computer is sent to the camera selector for camera selection. Based on the analysis above, the processing task occurs infrequently, only about dozens of times per second. So, the power consumption of the AMSM is mainly determined by the sensors including the accelerometer and the magnetometer. According to Table I and the actual sample rate, the power consumption of the accelerometer is about 30 , and that of the magnetometer is about 720 . The total power consumption of the AMSM can be less than 1 mW. IV. IMAGE COMPRESSION AND STORAGE For reducing the power consumption and increasing the image frame rate, the MCEC stores image data inside the flash memory module, instead of transmitting the image data wirelessly. The reason for this is that the energy efficiency of the flash memory module is higher than that of the wireless transceiver. The term ‘energy efficiency’ refers to the energy consumed by a device transmitting 1 bit of data. Further, the flash memory module has a higher data rate. On the basis of an actual measurement, a comparison between a flash memory chip and a wireless transceiver for WCE is shown in Table II. Furthermore, because of the use of the flash memory module, the patient need not carry a wireless data receiver when the MCEC is working in his GI tract. Compared with traditional WCE, this is more convenient and comfortable. The application of the image storage strategy can save the power consumption. However, from the power simulation anal-

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 8

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

TABLE II COMPARISON BETWEEN WIRELESS TRANSCEIVER AND FLASH MEMORY

TABLE III COMPARISON BETWEEN DIFFERENT TRANSFORMS

Fig. 13. Image compressor architecture. a: additions, m: multiplications, s:shifts

ysis, it is found that the power consumed by the image storage still accounts for approximately 40% of the total power consumption without image compression. Hence, the application of image compression before image storage can significantly reduce the total power consumption of the MCEC. Moreover, the image compression can reduce the flash memory capacity needed. Hence, it is very helpful for the miniaturization of the MCEC. Considering the tradeoff between the compression performance and the computation complexity, a modified JPEG image compression algorithm has been proposed. For low computational complexity and small buffer space, a new 4 4 Integer Block Transform (IBT-4) is proposed for the compression. The quantization table is optimized on the basis of the rate-distortion theory [29], and the elements of the quantization table are set to for eliminating the division operations. The compression algorithm is developed for Bayer Color Filter Array (CFA) pattern of the CMOS image sensor. R, G and B color components use the same quantization table to further reduce the hardware complexity. The Huffman encoding algorithm is selected for the entropy encoding after the quantization stage. A new integer transform matrix is proposed in (4). Its computation complexity is decreased by approximately 25%–40% in comparison with the other transforms as illustrated in Table III.

(4)

Hence, the optimized tained.

in (6) with the 40 dB constraint is ob-

(5) (6)

The architecture of the image compressor is shown in Fig. 13. It contains the image compressor controller, 4 4 block integer transform module , quantizer, Huffman encoder and configuration module including the quant table, huff table and config. With the configuration module, the quantization parameters and the Huffman table can be altered. The IBT-4 consists of two 1-D integer transform modules and a transpose buffer. The compressed image data are then stored inside the flash memory module. Fig. 14 shows the original WCE images with different patterns and the corresponding reconstructed images. Table IV shows that the proposed compression algorithm has low computation complexity and good compression performance when 50 typical WCE images containing different patterns are compressed. The computation complexity of ‘Before Encoding’ in Table IV is converted to ‘operations per pixel’. According to the experimental results in Section V, approximately 33% power consumption can be saved with the proposed image compressor. V. IMPLEMENTATION OF THE MCEC AND EXPERIMENTS A. Implementaion of the MCEC

The quantization table optimization is based on the ratedistortion theory. The distortion can be computed according to the PSNR definition. The compression rate can be computed by applying the Huffman encoding to the quantized block transform coefficients. Both and are determined by . The optimization process can be defined as (5). Considering the high compression rate and image quality, the search range of the varies from 4 to 64 ( , , 3, 4, 5, 6). By experimenting with different WCE images, we find that there are no visually discernible differences between the reconstructed image and the original image when the PSNR is higher than 40 dB.

The master chip and slave chip are fabricated in 0.18 1P6M technology. The die photo is shown in Fig. 15. The wireless transceiver contains more top metal than the digital core, so it is more legible in the die photo. Table V summarizes the main performance of the chips. The power consumption of the master chip and the slave chip under different conditions, including supply voltage and image frame rate, is measured and summarized in Table VI. It is noted that the power consumption in Table VI is that during the period of acquiring images inside human’s digestive tract. After the MCEC is evacuated, it is powered by wireless power transmission and starts image data

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. GU et al.: DESIGN OF ENDOSCOPIC CAPSULE WITH MULTIPLE CAMERAS

9

TABLE V CHIPSET PERFORMANCE SUMMARY

TABLE VI CHIPSET POWER CONSUMPTION

Fig. 14. Original and reconstruction WCE images.

TABLE IV COMPARISON BETWEEN DIFFERENT COMPRESSION ALGORITHMS

Fig. 16. Micro-ball prototype. a: additions, s:shifts

Fig. 15. Chipset die photo.

transmission. So, the power consumption restriction during the data transmission period is not so strict as that during the period of acquiring images. The wireless transceiver only works during the data transmission period. It can provide 3 Mbps MSK transmitting and 64 Kbps OOK receiving with power consumption of 3.9 mW and 12 mW respectively [16]. The wireless power

receiving chip has also been fabricated in 0.18 1P6M technology with a die area of 1.5 mm 1.5 mm [17]. It can provide stable DC power to the whole MCEC. For verifying the feasibility of the MCEC, a six cameras endoscopy prototype system has been implemented. A PCB cube with six cameras can be sealed into a transparent biocompatible plastic shell to form a ‘Micro-Ball’. It is shown in Fig. 16. The current size of the Micro-Ball is mainly determined by the flash memory chip size. If small size flash memory chip can be got, a more compact Micro-Ball can be implemented and be used for the GI tract endoscopy. The Data Reader uses an ARM processor as the controller. The WPT module and emitting coils are custom designed. It is shown in Fig. 17. The prototype system can be used for acquiring images. Fig. 18 shows the Micro-Ball prototype’s power consumption diagram of acquiring a frame of 480 480 image with the proposed image compression and without compression. The remarkable difference between the two situations is marked by the red circle. It can be seen that the number of program operations is significantly decreased after image compression. The

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 10

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

Fig. 19. Scheme of proposed 2D representation method.

Fig. 17. Data reader.

Fig. 20. Coverage rate experiment environment.

TABLE VIII ACTUAL AND SIMULATION SITUATIONS OF MICRO-BALL’S MOVEMENT

Fig. 18. Power consumption diagram.

TABLE VII BREAKDOWN OF ENERGY CONSUMPTION

total energy consumption can be reduced by about 33%. The energy consumed for acquiring a frame of 480 480 image with the proposed image compression is only about 2.6 mJ. The main constituent parts of the total energy consumption are shown in Table VII. B. Smart Image Capture For evaluating the coverage rate of the images captured by the MCEC, an image post-processing method is applied to obtain an integrated 2D representation of all the captured images. The scheme of this method, as shown in Fig. 19, includes three main stages: 1) Image preprocessing based on color space and image enhancement technologies; 2) rectification of perspective

distortions; and 3) Image registration based on SIFT and improved Phase Correlation Method (PCM). In order to realize robust image registration, the improved PCM proposed in our previous work [30] and SIFT [31] are adopted in this study to first judge whether the two images can be registered or not and then obtain the registration parameters. As shown in Fig. 20, in order to evaluate the effectiveness of the smart image capture strategy, the Micro-Ball is placed into a flexible pipe and images are captured while the Micro-Ball is moving. The inner wall of the flexible pipe is covered with a reference picture. According to [20]–[23] and the observation of the WCE image sequences of different patients, the actual movement of the Micro-Ball inside the small bowel is estimated. The rotation occurrence frequency is approximately deduced from the perspective changes of the WCE image sequence. The simulation movement is obtained by expanding some parameters in proportion to the actual size with the prototype system size. The two situations are presented in Table VIII. When the Micro-Ball is moved according to Table VIII, the AMSM is working for obtaining the motion and attitude information. Considering the motion frequency and low power consumption requirement, the sample rate of the accelerometer is set to 50 sps (samples per seconds) and that of the magnetometer is set to 24 sps. After sensor calibration, the results of DCM computation reflected in Euler angles, including pitch, roll and yaw, are shown in Fig. 21(a), (b) and (c). The errors of pitch, roll and yaw are 3.4 , 2.2 and 5.1 , respectively. Considering

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. GU et al.: DESIGN OF ENDOSCOPIC CAPSULE WITH MULTIPLE CAMERAS

11

This situation is similar to that of the traditional wireless endoscopic capsule. The images captured using the different strategies have been post-processed by the proposed method in order to obtain a 2D representation. Fig. 22(a) shows the 2D representation result under smart image capture control mode, and Fig. 22(b) shows the reference picture. The coverage rate covr is defined as (8). Table IX shows the average results from 20 sets of experiments. Compared with the six cameras and two cameras without the smart image capture control, the smart image capture has the highest coverage rate. Although the experiment environment above is not the same as the actual situation in human’s GI tract, the experimental results can prove the effectiveness of the smart image capture strategy for increasing the coverage rate of the MCEC in principle. The number of images captured under different modes shown in Table IX indicates that the smart image capture strategy could reduce redundancy of the captured images.

(8)

Fig. 21. DCM computation results. (a) Pitch versus time. (b) Roll versus time. (c) Yaw versus time.

that one camera’s field of view is wider than 90 , the accuracy of DCM computation is high enough for the camera selection. Considering the wide coverage area and the low power consumption requirements, the parameters of the image frame rate control mechanism are set as (7).

The power consumption and hardware resources of the MCEC working at the fixed image frame rates and under the smart capture control are also given in Table IX. By increasing number of cameras, the coverage rate can be increased by 6% at most. In this situation, only the number of hardware resources is increased and the power consumption increasing is very a little because of the power control of the idle module. By increasing image acquisition rate, coverage rate can be increased by 10% at most. However, it leads to a very high power consumption of 70 mW more. Although the smart capture control introduces extra hardware resources (1 accelerator and 1 magnetometer) and power consumption (only 0.83 mW), it can improve coverage rate and save total power consumption significantly. Under the smart image capture mode, the MCEC can work for more than 12 hours at an image resolution of 480 480, when powered by two 1.55 V@50 mAh batteries. This battery life is long enough for the diagnosis of the entire human digestive tract. VI. CONCLUSION

(7) Applying the smart image capture strategy, the Micro-Ball captures images while moving. For the sake of comparison, all the cameras inside the Micro-Ball are set to capture images in turns, and the image frame rate is fixed (6 fps and 24 fps) in another experiments. We also choose two cameras to capture images at a fixed image frame rate (6 fps and 24 fps). The orientations of the two cameras are the axial directions of the pipe.

In this paper, we proposed a smart MCEC for medical endoscopy applications. With multiple cameras embedded in the MCEC and smart image capture control, the MCEC can cover a relatively large area of the GI tract wall. With the proposed efficient master-slave architecture and low complexity image compressor, the power consumption can be significantly reduced. The energy consumed for acquiring a frame of 480 480 image is only about 2.6 mJ. The coverage rate of the capsule prototype can achieve 98% under the smart image capture mode and the power consumption is only about 7. 1 mW. The above mentioned designs enable the MCEC to efficiently carry out the diagnosis of the entire human digestive tract.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 12

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

Fig. 22. 2D representation result. (a) 2D representation result. (b) Reference picture.

TABLE IX PERFORMANCE UNDER DIFFERENT MODES

REFERENCES [1] G. Iddan, G. Meron, and A. Glukhovsky et al., “Wireless capsule endoscopy,” Nature, vol. 405, pp. 417–418, May 25, 2000. [2] T. Nakamura and A. Terano, “Capsule endoscopy: Past, present, future,” J. Gastroenterol., vol. 43, no. 2, pp. 93–99, 2008. [3] S. Bar-Meir and M. B. Wallace, “Diagnostic colonoscopy: The end is coming,” Gastroenterol., vol. 131, pp. 992–994, 2006. [4] N. Schoofs, J. Deviere, and A. Van Gossum, “Pillcam colon capsule endoscopy compared with colonoscopy for colorectal tumor diagnosis: A prospective pilot study,” Endoscopy, vol. 38, pp. 971–977, 2006. [5] P. Demosthenous and J. Georgiou, “Acceleration-dependent sampling for ingestible endoscopic imaging capsule,” in Proc. IEEE Biomedical Circuits and Systems Conf., Nov. 28–30, 2012, pp. 1–4. [6] S. Tognarelli, C. Quaglia, P. Valdastri, E. Susilo, A. Menciassi, and P. Dario, “Innovative stopping mechanism for esophageal wireless capsular endoscopy,” in Proc. 23rd Eurosensors Conf., 2009, pp. 485–488. [7] P. Glass, E. Cheung, and M. Sitti, “A legged anchoring mechanism for capsule endoscopes using micropatterned adhesives,” IEEE Trans. Biomed. Eng., vol. 55, no. 12, pp. 2759–2767, Dec. 2007.

[8] G. Tortora, P. Valdastri, E. Susilo, A. Menciassi, P. Dario, F. Rieber, and M. Schurr, “Propeller-based wireless device for active capsular endoscopy in the gastric district,” Minim. Invasive Therapy Allied Technol., vol. 18, pp. 280–290, 2009. [9] A. Uehara and K. Hoshina, “Capsule endoscope Norika system,” Minim. Invasive Therapy Allied Technol., vol. 1, pp. 227–234, 2003. [10] Y. Kusuda, “A further step beyond wireless capsule endoscopy,” Sens. Rev., vol. 25, pp. 259–260, 2005. [11] C. Yu, J. Kim, H. Choi, J. Choi, S. Jeong, K. Cha, J.-O. Park, and S. Park, “Novel electromagnetic actuation system for three-dimensional locomotion and drilling of intravascular microrobot,” Sens. Actuators A, Phys., vol. 161, pp. 296–304, 2010. [12] J. F. Rey, H. Ogata, N. Hosoe, K. Ohtsuka, N. Ogata, K. Ikeda, H. Aihara, I. Pangtay, T. Hibi, S. Kudo, and H. Tajiri, “Feasibility of stomach exploration with a guided capsule endoscope,” Endoscopy, vol. 42, pp. 541–545, 2010. [13] X. Chen, X. Zhang, L. Zhang, X. Li, N. Qi, H. Jiang, and Z. Wang, “A wireless capsule endoscope system with low-power controlling and processing ASIC,” IEEE Trans. Biomed. Circuits Syst., vol. 3, no. 1, pp. 11–22, 2009.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. GU et al.: DESIGN OF ENDOSCOPIC CAPSULE WITH MULTIPLE CAMERAS

[14] T. H. Khan and K. Wahid, “Low-complexity colour-space for capsule endoscopy image compression,” Electron. Lett., vol. 47, no. 22, pp. 1217–1218, 2011. [15] W. D. Toh, B. Zhao, Y. Gao, M. Je, and C. H. Heng, “A low power image sensor controller and JPEG encoder IC for wireless capsule endoscopy,” Int. J. Inf. Elect. Eng., vol. 3, no. 1, pp. 32–34, 2013. [16] H. Jiang, F. Li, and Z. Wang et al., “A SoC with 3.9 mW 3 Mbps UHF MCU for capsule endoscope with bidirectransmitter and 240 tional communication,” in Proc. IEEE Asian Solid State Circuits Conf., pp. 1–4. [17] T. Sun, X. Xie, G. Li, Y. Gu, X. Li, and Z. Wang, “An omnidirectional wireless power receiving IC with 93.6% efficiency CMOS rectifier and Skipping Booster for implantable bio-microsystems,” in Proc. IEEE Asian Solid State Circuits Conf., Nov. 14–16, 2011, pp. 185–188. [18] X. Wang, M.-H. Meng, and Y. Chan, “Physiological factors of the small intestine in design of active capsule endoscopy,” in Proc. 27th Annu. Int. Conf. Engineering in Medicine and Biology Soc., 2006, pp. 2942–2945. [19] A. C. Ailiani, T. Neuberger, J. G. Brasseur, G. Banco, Y. Wang, N. B. Smith, and A. G. Webb, “Quantitative analysis of peristaltic and segmental motion in vivo in the rat small intestine using dynamic MRI,” Magn. Reson. Med., vol. 62, no. 1, pp. 116–126, 2009. [20] J. M. Froehlich, M. A. Patak, C. von Weymarn, C. F. Juli, C. L. Zollikofer, and K.-U. Wentz, “Small bowel motility assessment with magnetic resonance imaging,” J. Magn. Reson. Imag., vol. 21, no. 4, pp. 370–375, 2005. [21] K. Hipper and H. J. Ehrlein, “Motility of the large intestine and flow of digesta in pigs,” Res. Veterin. Sci., vol. 71, no. 2, pp. 93–100, 2001. [22] N. K. Baek, I. H. Sung, and D. E. Kim, “Frictional resistance characteristics of a capsule inside the intestine for microendoscope design,” Proc. Inst. Mech. Eng., Part H: J. Eng. Med., vol. 218, no. 3, pp. 193–201, 2004. [23] J.-S. Kim, I.-H. Sung, Y.-T. Kim, E.-Y. Kwon, D.-E. Kim, and Y. H. Jang, “Experimental investigation of frictional and viscoelastic properties of intestine for microendoscope application,” Tribol. Lett., vol. 22, no. 2, pp. 143–149, 2006. [24] M. D. Shuster and S. D. Oh, “Three-axis attitude determination from vector observations,” J. Guid. Control Dyn., vol. 4, no. 1, pp. 70–77, 1981. [25] B. Huyghe, P. Salvo, J. Doutreloigne, and J. Vanfleteren, “Feasibility study and performance analysis of a gyroless orientation tracker,” IEEE Trans. Instrum. Meas., vol. 61, no. 8, pp. 2274–2282, 2012. [26] D. Gebre-Egziabher, G. H. Elkaim, J. D. Powell, and B. W. Parkinson, “A gyro-free quaternion-based attitude determination system suitable for implementation using low cost sensors,” in Proc. IEEE Position Location and Navigation Symp., 2000, pp. 185–192. [27] A. Mostafa, K. Wahid, and S.-B. Ko, “A low-power subsample-based image compression algorithm for capsule endoscopy,” in Proc. IEEE Int. Symp. Circuits and Systems, 2012, pp. 109–112. [28] L.-R. Dung, Y.-Y. Wu, H.-C. Lai, and P.-K. Weng, “A modified H. 264 intra-frame video encoder for capsule endoscope,” in Proc. IEEE Biomedical Circuits and Systems Conf., 2008, pp. 61–64. [29] A. C. Hung and T.-Y. Meng, “Optimal quantizer step sizes for transform coders,” in Proc. Int. Conf. Acoustics, Speech, Signal Processing, 1991, pp. 2621–2624. [30] D. Wang, X. Xie, G. Li, Y. Gu, T. Sun, and Z. Wang, “Image registration method for 2D representation of wireless Micro-Ball endoscopic images,” in Proc. IEEE Int. Symp. Circuits and Systems, 2011, pp. 93–96. [31] D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” Int. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004. [32] J. M. Herrerias, A. Caunedo, M. Rodriguez-Tellez, F. Pellicer, and J. M. Herrerias Jr, “Capsule endoscopy in patients with suspected Crohn’s disease and negative endoscopy,” Endoscopy, vol. 35, no. 7, pp. 564–568, 2003. [33] L.-R. Dung and Y.-Y. Wu, “A wireless narrowband imaging chip for capsule endoscope,” IEEE Trans. Biomed. Circuits Syst., vol. 4, no. 6, pp. 462–468, Dec. 2010. [34] P. Swain, G. J. Iddan, G. Meron, and A. Glukhovsky, “Wireless capsule endoscopy of the small bowel: Development, testing, first human trials,” in Proc. EOS/SPIE Eur. Biomedical Optics Week, 2001, pp. 19–23. [35] K. Shiba, A. Morimasa, and H. Hirano, “Design and development of low-loss transformer for powering small implantable medical devices,” IEEE Trans. Biomed. Circuits Syst., vol. 4, no. 2, pp. 77–85, Apr. 2010.

13

[36] A. Glukhovsky and H. Jacob, “The development and application of wireless capsule endoscopy,” Int. J. Med. Robot. Comput. Assist. Surgery, vol. 1, no. 1, pp. 114–123, 2004. [37] A. Kolar, O. Romain, J. Ayoub, S. Viateur, and B. Granado, “Prototype of video endoscopic capsule with 3-D imaging capabilities,” IEEE Trans. Biomed. Circuits Syst., vol. 4, no. 4, pp. 239–249, Aug. 2010. [38] C. Ell, S. Remke, A. May, L. Helou, R. Henrich, and G. Mayer, “The first prospective controlled trial comparing wireless capsule endoscopy with push enteroscopy in chronic gastrointestinal bleeding,” Endoscopy, vol. 34, no. 09, pp. 685–689, 2002. [39] M. Mylonaki, A. Fritscher-Ravens, and P. Swain, “Wireless capsule endoscopy: A comparison with push enteroscopy in patients with gastroscopy and colonoscopy negative gastrointestinal bleeding,” Gut, vol. 52, no. 8, pp. 1122–1126, 2003.

Yingke Gu received the B.S. and M.S. degrees from Tsinghua University, Beijing, China, in 2004 and 2008, respectively. Currently, he is working toward the Ph.D. degree at Tsinghua University. His research focuses on low power VLSI design, image processing, and wireless capsule endoscopy system design.

Xiang Xie is an Associated Professor at the Institute of Microelectronics, Tsinghua University, Beijing, China. His research fields cover SoC design, image processing, biomedical electronics, and pervasive HCI. His current main research projects include a new generation of wireless Micro-Ball endoscopy system, portable and mobile 3D image data acquisition and processing, and SoC and DSP design for the pervasive HCI system. He is the author or coauthor of more than 80 papers, two books, and one book chapter in the related research fields. He has filed over 10 patents and applied for over 10 patents in this research field.

Guolin Li received the B.S., M.S., and Ph.D. degrees from the Department of Electronics Engineering, Tsinghua University, Beijing, China, in 1993, 1998, and 2002, respectively. Currently, he is an Associate Professor at Tsinghua University. His current research interests include the circuit design of RFIC, WPT, etc.

Tianjia Sun Photograph and biography not available at the time of publication.

Dan Wang Photograph and biography not available at the time of publication.

Zheng Yin Photograph and biography not available at the time of publication.

Pengfei Zhang Photograph and biography not available at the time of publication.

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. 14

Zhihua Wang (M’99–SM’04) received the B.S., M.S., and Ph.D. degrees in electronic engineering from Tsinghua University, Beijing, China, in 1983, 1985, and 1990, respectively. In 1983, he joined the faculty at Tsinghua University, where he is a Full Professor since 1997 and Deputy Director of Institute of Microelectronics since 2000. From 1992 to 1993, he was a Visiting Scholar at Carnegie Mellon University, Pittsburgh, PA, USA. From 1993 to 1994, he was a Visiting Researcher at KU Leuven, Belgium. His current research mainly focuses on CMOS RF IC and biomedical applications. His ongoing work includes RFID, PLL, low-power wireless transceivers, smart clinic equipment with combination of leading edge CMOS RFIC, and digital imaging processing techniques. He is coauthor of 10 books and book chapters, more than 90 papers in international journals and over 300 papers in international conferences. He holds 58 Chinese patents and four U.S. patents. Dr. Wang has served as Deputy Chairman of the Beijing Semiconductor Industries Association and ASIC Society of Chinese Institute of Communication,

IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS

as well as Deputy Secretary General of the Integrated Circuit Society in the China Semiconductor Industries Association. He had been one of the chief scientists of the China Ministry of Science and Technology, serving on the expert committee of the National High Technology Research and Development Program of China (863 Program) in the area of information science and technologies from 2007 to 2011. He had been an official member of China Committee for the Union Radio-Scientifque Internationale (URSI) from 2000 to 2010. He was the Chairman of the IEEE Solid-State Circuit Society Beijing Chapter from 1999 to 2009. He served as a technologies program committee member of the IEEE International Solid-State Circuit Conference (ISSCC) from 2005 to 2011. He has been a steering committee member of the IEEE Asian Solid-State Circuit Conference (A-SSCC) since 2005 and has served as the Technical Program Chair for the 2013 A-SSCC. He served as a Guest Editor for IEEE JOURNAL OF SOLID-STATE CIRCUITS Special Issue in December 2006, December 2009, and November 2014. He is an Associate Editor for IEEE TRANSACTIONS ON BIOMEDICAL CIRCUITS AND SYSTEMS and IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS—PART II: EXPRESS BRIEFS.

Design of Endoscopic Capsule With Multiple Cameras.

In order to reduce the miss rate of the wireless capsule endoscopy, in this paper, we propose a new system of the endoscopic capsule with multiple cam...
3MB Sizes 31 Downloads 6 Views