Luc Claesen

Peter Vandoren

Tom Van Laerhoven

Andy Motten

Fabian Di Fiore

Frank Van Reeth

Jing Liao $

Jinhui Yu
email: jhyu@cad.zju.edu.cn

Jing Liao$
email: liaojing@cad.zju.edu.cn

Smart Camera System-on-Chip Architecture for Real-Time Brush Based Interactive Painting Systems

Keywords: System-on-Chip, SoC, FPGA, digital painting, active canvas, animation, cartoons, artwork, editing, Video Processing, HCI, human-computer interface, real-time, Smart Camera, embedded video, FTIR, video pipeline

Interactive virtual paint systems are very useful in editing all kinds of graphics artwork. Because of the digital tracking of strokes, interactive editing operations such as save, redo, resize etc. are possible. The structure of artwork generated can be used for animation in artwork cartoons. A novel System-on-Chip Smart Camera architecture is presented that can be used for tracking infrared fiber based brushes as well as real brushes in real-time. A dedicated SoC hardware implementation avoids unnecessary latency delays caused by PC based architectures, that require communication-, PC and GPU frame-buffer delays, thereby considerably enhancing the interactivity experience. The system is prototyped on an FPGA.

Introduction

During the centuries several kinds of art and graphics representations have been in use up till today. Most of the artwork is still produced by means of traditional tools such as pencil and paint-brushes. Since the introduction of graphic computer displays, software painting applications have been developed [START_REF] Smith | Digital Paint Systems: An Anecdotal and Historical Overview[END_REF]. Most current computers provide software where simple paintings can be made by means of a computer mouse, a touch screen or a stylus/tablet input device. Usually a color bit-map file is generated by the input device by adding and combining one or more bit-map layers of the canvas and the virtual brush at the mouse pointer or pen-stylus cursor.

The most widely available paint systems make use of the "rubber stamp principle". This means that a specific two-dimensional image "a stamp", with texture, size and color is attached to the mouse or stylus cursor. When moving the mouse or drawing pen over the canvas, the image of the "rubber stamp" is added to the background bitmap image.

The use of computers in art-and graphics image creation has several advantages. The artwork can be saved for later retrieval. It is also possible to edit the artwork to adapt it better to the ideas of the artist or graphics designer. Mistakes can be coped with in an easier way by means of "undo" operations. Parts of designs can be selected, copied and pasted to generate other images. When individual strokes are recorded and represented as parametric curves, they can be manipulated afterwards by the user, to adapt the shape and size. An even further step is the composition of artwork for use in the generation of moving images and cartoons [START_REF] Yu | Modeling the interaction between objects and cartoon water[END_REF][START_REF] Di Fiore | Highly Stylized Animation[END_REF].

Besides simple "rubber-stamping" based paint systems more complex models have been developed over the years as discussed next.

1.1

Active-canvas digital paint methods.

More detailed models for digital painting have recently been developed which attempt to mimic the real painting process using detailed simulation of the complex interaction between brushes and the paint canvas. These are so-called "active-canvas" methods. They model the paint as a solvent fluid that can flow and evaporate and that contains color pigments and a fixation binder glue. The pigments are small 10..150nm particles having their own dispersion and color characteristics. The canvas is characterized by its non-uniform texture and solvent/paint absorption characteristics.

Curtis e.a. [START_REF] Curtis | Computergenerated watercolor[END_REF] introduce an empirically based watercolor paint system that is based on an ordered set of translucent glazes. Each wash is simulated by three layers: a shallow-water layer, a pigment deposition layer and a capillary layer where the water is absorbed by and diffused in the paper. Hereby all kinds of water-color effects such as wet-on-wet and wet-on-dry, dry-brush effects, edge darkening, backruns, granulation and separation of pigments, flow patterns etc. can be simulated. This process is based on the finite element simulation of the Navier-Stokes fluid-dynamics equations, Poisson diffusion equations and Kubelka-Munk color model. The IMPaSTo system [START_REF] Baxter | IMPaSTo: A realistic, interactive model for paint[END_REF] specifically models paint methods such as oils, acrylics or gouache, based on a conservative advection scheme that simulates the basic dynamics of paint. These physical based painting models are computationally very intensive, posing a problem for real-time execution. Van Laerhoven e.a. [START_REF] Van Laerhoven | Real-time simulation of watery paint[END_REF] realized new algorithms suited for the real-time execution on GPUs (Graphics Processing Units). The above methods [START_REF] Curtis | Computergenerated watercolor[END_REF][START_REF] Baxter | IMPaSTo: A realistic, interactive model for paint[END_REF][START_REF] Van Laerhoven | Real-time simulation of watery paint[END_REF] are all based on differential equations like Navier-Stokes and Poisson, describing the macroscopic effects of the paint dynamics. Numerical integration methods of the differential equations based on forward Euler can result in unstable behavior in case of fast changes (fast brush strokes). Backward Euler based methods on the other hand, although stable, require a number of iterations, adding to the computational complexity. To cope with these problems, Chu e.a. [START_REF] Chu | MoXi: Real-Time Ink Dispersion in Absorbent Paper[END_REF] introduced the use of the Lattice-Boltzmann equations for paint-simulation, modeling the physics of the movement of paint particles at a mesoscopic level. This method is implemented on GPU's and can model the physics of ink flow in absorbent paper.

A massively parallel hardware implementation of the Lattice-Boltzmann model for active canvas simulation has recently been prototyped making use of FPGA's [START_REF] Nowicki | SoC Architecture for Real-Time Interactive Painting based on Lattice-Boltzmann[END_REF].

1.2

Artist-Computer Interfaces for Digital Paint systems.

To create digital paintings, a computer mouse can be used. Although computer mice have proven their usefulness in a lot of daily computer tasks, the expressiveness for digital painting is rather limited. Computer mice only record relative movements and have little expressiveness for pressure input. Therefore professional artists currently prefer tablets and stylus pens. These systems have a good absolute accuracy with respect to the drawing tablet and also provide a measurement of the drawing force along the axis of the pen stylus shaft. This allows paint programs to model the force exercised on the pen tip while drawing and consequently generating thinner or thicker pen strokes depending on the force employed by the artist.

Mueller [START_REF] Mueller | Direct Television Drawing and Image Manipulation System[END_REF] describes a real-time painting system based on frustration of internal reflected light in a prism. The light in the prism is generated, via an optical setup, by the scan signal of a CRT (Cathode Ray Tube) image. The frustrated light generated by a drawing utensil can be detected by a (photo multiplier) light sensor in a synchronous way with the CRT scan signal. This enables a quasi real-time brush detection. The rendering of the painting result on the screen is not co-located with the drawing surface.

Greene [START_REF] Greene | The Drawing Prism: A Versatile Graphic Input Device[END_REF] introduced the drawing prism, commercialized under the name OptiPaint [7]. This is a drawing system built around a prism. One side of the prism is used as a drawing surface. Wet brushes and other drawing tools can be detected by means of frustration of a light source which is normally reflected on the drawing surface side of the prism by means of total internal reflection. The frustrated light caused by the contact of drawing tools is captured by means of a video camera located at an other side of the drawing prism. This system requires a setup with a bulky optically transparent prism of which one side is at least the size of the drawing surface and the other side is at least large enough for enabling a good imaging by a video camera. The aspect ratio of the camera side deviates from the standard aspect ratios of video cameras, thus requiring a special optical setup or camera design. The rendering of the painting result on the screen is separate from the drawing surface [START_REF] Greene | The Drawing Prism: A Versatile Graphic Input Device[END_REF]7] just like the method of Mueller [START_REF] Mueller | Direct Television Drawing and Image Manipulation System[END_REF]. This means that a user is drawing or painting on a tablet or surface where he/she does not directly see anything of his/her artwork. The coordination of drawing on one surface and viewing the result on a separate screen requires special training and concentration. It is not intuitive to users. Carver Mead et al. [START_REF] Mead | Paintbrush stylus for capacitive sensor pad[END_REF] proposed a paintbrush stylus sensed by a capacitive sensor array. Because of the resolution of the capacitive sensors the main input parameters are the coordinates. Because of the capacitive sensing mechanism only electrically conductive brushes can be used in this system. The capacitive sensor technology is what is used in current tablet computers and smart phones such as the Apple iPad and iPhone.

Electro-magnetic tablets are probably the most widely used input devices for paintsystems. In Wacom tablets, in one phase, two orthogonal grids of wires generate alternating and localized electromagnetic fields on the tablet. This transfers some energy to an LC tank inside the stylus pen. This energy is used in an other phase to generate an electromagnetic field from the pen that can than be sensed by the same orthogonal grid of wires in the tablet. These tablets can generate an accurate stylus position and can also often sense the one dimensional force in the direction of the stylus axis. By using a layer of optically transparent wires on top of an LCD display, the Wacom Cintic system [START_REF]Wacom Cintic[END_REF] integrates the input tablet with the drawing screen. This provides direct feedback of the drawing result under the pen tip. Because tablet based systems use stiff styluses, painting with a stiff stylus is different than painting with real brushes with flexible tufts made of camel, hog, squirrel hair or other natural or synthetic fibers. In Western painting and Chinese calligraphy, the specific movement and deformation of the brush tuft is crucial for achieving special effects. Although some tablet systems [START_REF]Wacom Cintic[END_REF] provide co-located drawing input/painting display, they suffer from the distance between the drawing plane and the display plane which causes a parallax effect. Depending on the relative position of the artists, the pen tip and its drawing result on the screen will be different. The IntuPaint system [START_REF] Vandoren | IntuPaint: Bridging the Gap Between Physical and Digital Painting[END_REF] uses electronic brushes with bristles made of optically transparent fibers (Fig. 1.). An infrared light source inside the brush propagates light through the transparent fibers by means of total internal reflection. The light exits at the bristle tips. When IntuPaint brushes are in contact with a diffuser screen, the tuft footprint and position can be imaged by an infrared camera behind the screen. In IntuPaint, the diffuser screen is used to display the result of painting, thereby providing co-located input/display. By using a brush with bristles, an artist can exploit the deformation of the brush tuft during drawing by brush movements, inclination and pressure on the canvas. Fig. 2. shows the system in use. Because of its infrared light emission operating principle, the IntuPaint system requires specially built brushes and drawing tools. All previous methods still limit artists in their expressiveness in comparison to traditional painting with brushes and paint. To solve this problem, the authors have introduced the FluidPaint system [START_REF] Vandoren | FluidPaint: an Interactive Digital Painting System using Real Wet Brushes[END_REF]. Both the IntuPaint and the FluidPaint system are built on top of physical based painting simulation software [START_REF] Van Laerhoven | Real-time simulation of watery paint[END_REF] running on a highend GPU powered PC. The novelty of FluidPaint is that it uses real brushes on a colocated painting input/display canvas surface. To enable the real-time and low-latency virtual painting, a dedicated Smart Camera based SoC architecture has been developed and is presented in this paper.

In Section II, the system setup of the FluidPaint virtual painting system with real brushes is presented. In Section III the usage of the Smart Camera is introduced. The Smart Camera hardware architecture and prototype is presented in Section IV. Section V formulates conclusions and further work.

2

Virtual Painting With Real-Brushes.

FluidPaint is a novel digital painting system that operates with real brushes. In this section the operation principle of FluidPaint is briefly described. The reader is referred to [START_REF] Vandoren | FluidPaint: an Interactive Digital Painting System using Real Wet Brushes[END_REF] for a more in-depth presentation and user tests. The FluidPaint paint canvas constitutes the key component of the system. It is a 3layered system as shown in Fig. 3.

Layer 1 consists of a 0.6mm thick transparent surface layer. On the four sides of the transparent layer there is an array of 950nm infrared (IR) LEDs, introducing IR light inside the transparent layer, which acts as an optical waveguide. This IR light is propagated in the layer by means of total internal reflection and normally exits the layer at the other side.

Layer 2 consists of a diffuser screen. A projector positioned below the paint canvas can project an image of a painting on the diffuser screen (Fig. 2).

Layer 3 is a transparent support layer giving mechanical strength to the drawing surface. When a wet brush makes contact with the top layer, as illustrated in Fig. 3, the IR light inside the top layer is not internally reflected anymore and can propagate outside the layer and propagate inside the water in the wet brush until it arrives at the brush bristles. Here the IR will be scattered in different directions according to the bristle structure. An IR camera placed below the screen can capture this IR image. It is in fact a footprint of the brush contact surface as illustrated by brush B in Fig. 6.. When a dry brush is put into contact with the surface layer, there is nearly no optical contact and consequently the light inside Layer 1 remains internally reflected and is not frustrated. Consequently no image is visible by the IR camera as is illustrated by brush A in Fig. 6.. When using wet brushes, wet traces are left on the drawing surface. As shown in Fig. 3 This input method of painting with real and wet brushes results in a feeling and expressiveness like in real-world painting. The feeling and brush-hand feedback to the artist is similar as when painting with real paint. The IR camera only images the brush contact surface. During real painting it is also only the contact surface that really matters. The image of the contact surface images the real brush and bristle structure in the contact zone. Such a brush footprint can be very well used in physical model based painting systems [START_REF] Van Laerhoven | Real-time simulation of watery paint[END_REF]. This enables an artist to express very small nuances due to the specific brush movements and complex tuft deformation during the act of painting. The painting input and rendered painting canvas display is co-located. The artist directly sees the result of the painting under the brush as illustrated in Fig. 7. In Fig. 8 the bristle structure dependent brush stroke output and paint result is visible.

Although also based on the principle of total internal reflection, multi-touch systems as introduced by Han [START_REF] Han | Low-cost multi-touch sensing through frustrated total internal reflection[END_REF] are not directly usable for painting like the 3-Layer structure of FluidPaint. These multi-touch systems usually consist of a ~1cm thick transparent acrylic layer with IR leds on the side. On top of this acrylic layer there is a "compliant layer", usually made of silicone. On top of the compliant layer is a diffuser screen. Touch is detected by the IR light frustration of the contact points of the middle silicone with the bottom acrylic layer. Using a brush on such a system requires an unnatural high force from the brush to make contact through the diffuser screen and through the silicone layer with the acrylic layer. No detailed footprint as in FluidPaint is possible with such a system. Multi-touch systems without compliant layer have also been realized. They consist of a ~1cm thick acrylic layer with a diffuser screen under it. Although here wet brushes could be used like in FluidPaint, the distance (~1cm) between the brush contact surface and the diffuser screen is too high, resulting in a very unclear and blurred image below the diffuser screen and would also reintroduce an undesired parallax.

Smart Camera Usage.

The constant evolution in VLSI technology expressed by Moore's Law is an enabler for the complete integration of Systems-on-Chip (SoC). Because of the use of compatible silicon technologies, several applications already integrate CMOS cameras and image processing in Smart Cameras [START_REF] Wolf | Smart Cameras as Embedded Systems[END_REF] on the same SoCs [START_REF] Ruedi | An SoC combining a 132db QVGA pixel array and a 32b DSP/MCU processor for vision applications[END_REF]. A widely used Smart Camera SoC is the PixArt infrared blob-position detector in the Nintendo Wii remote controller. In this section the usage of a Smart Camera in the FluidPaint digital painting system is presented.

The first prototype of the FluidPaint system [START_REF] Vandoren | FluidPaint: an Interactive Digital Painting System using Real Wet Brushes[END_REF] made use of a standard machine vision camera. It was a PointGrey GRAS-20S4C camera with an IEEE 1394b FireWire interface to the host PC. Using standard cameras has the advantage of fast prototyping. The disadvantage is however that in applications such as digital painting, there are very stringent real-time requirements, both on the overall processing time as well as on the latency between brush input and processed display reaction. A standard machine vision camera sends full images to the PC, where further image processing is to be done to detect the brush footprint images and positions. It is well known that streaming video data and real-time image processing are very computation intensive. Delays in a traditional video pipeline occur in the camera, the transmission via Cameralink, Firewire or Ethernet, the capture in the receiving PC, the preprocessing, the transfer from the PC memory to the GPU memory and the displaying. In addition, standard cameras add delays between the capture of the image in the camera sensor and the delivery of the image processed results to the painting application. This delay which usually consists of several frame periods, causes a latency between painting with a brush and displaying the result on the screen. This is noticed by the fact that the paint on the canvas screen does not immediately follow the brush movements.

As the application PC is already very occupied with the paint simulation software, the combination with the camera image streaming communication and processing limits the real-time simulation effects.

A Smart Camera SoC architecture can 1) perform the required image processing in hardware and 2) reduce the delay time from image capture to processing. Implementing the image processing in dedicated hardware relieves the host PC from a compute intensive part, but also allows to reduce the required communication to the brush position and footprint only. The direct processing of the image data in hardware can avoid the use of unnecessary frame buffers in the camera and the PC. In a SoC, frame buffers can be reduced to the absolute minimum and can directly be employed for the required image processing at hand. In case of a controlled environment lighting, frame buffers could even be avoided.

An SoC architecture also allows for a direct per-frame camera control without lost frames. Hereby the SoC can directly change the camera field of view, shutter times, gains, black level calibration etc.

3

Smart Camera SoC Architecture.

Image Processing Pipeline

An infrared camera captures the image of the contact of the wet brush with the canvas as shown in Fig. 6. Image processing [START_REF] Rafael | Digital Image Processing[END_REF][START_REF] Kenneth | Digital Image Processing[END_REF] and segmentation enable the accurate determination of the brush location on the canvas and the determination of the detailed brush footprint image. The segmented brush footprint image is the input for the physical model-based paint simulation software [START_REF] Van Laerhoven | Real-time simulation of watery paint[END_REF].

The image processing consists of the following actions:

• Preprocessing of the sensor data.

• 5x5 Low pass Gaussian filter.

• Background subtraction.

• 5x5 Population thresholding.

• Contrast enhancement.

• Segmented Footprint Center of Gravity Determination.

• Brush footprint image identification.

• Camera/Screen image rectification.

• Transmission to PC application.

The first steps in the image processing isolate and enhance the image of the footprint. The location of the footprint could be determined by its edges or its horizontal or vertical histograms. In order to obtain a more accurate and stable position determination, the center of gravity of the footprint is determined. The footprint image around this center of gravity is transmitted to the painting application on the PC. Painting applications can make use of this property.

The co-location of the brush input/canvas screen requires a transformation of camera coordinates to screen coordinates of the projector. Careful placement of the mirrors M1 and M2 (see Figure 2.) can generate a rectangular projector image corresponding to the canvas screen. It is however very difficult to position the camera so that its canvas field of view corresponds to the projector canvas field of view. Distortions due to the other placement of the camera, due to lens distortions (cushion effect), due to different pixel densities etc. need to be compensated. This camera image rectification is done by a grid of calibrated control points in which camera coordinates are transformed to projector coordinates by means of bilinear transformation [START_REF] Kenneth | Digital Image Processing[END_REF]. Fig. 9 illustrates the Smart Camera architecture. The architecture consists of a programmable interconnect fabric that allows the flexible arrangement of image processing operations in a pipeline. This architecture supports the operators required by the Image Processing Pipeline: camera preprocessing, lowpass Gaussian filtering, Background subtraction, Population based Thresholding, Contrast enhancement, Center of Gravity Calculation, Histogram calculation. The 5x5 lowpass filter and the 5x5 population thresholding operators use on-chip line buffer memories. Two independent DRAM frame buffer based memories can be used. A first frame buffer stores the background image for background subtraction. This background image is used for an adaptive background determination by means of a first order IIR (Infinite Impulse Response) temporal image filter. This can be interesting in environments with (slowly) changing infrared background lighting. A second frame buffer is used to store the incoming image. After the location of the brush has been determined, the brush position and footprint image are sent to the host PC via a direct Ethernet link.

The image sensor camera and all of the image processing operators and communication are controlled by a 32 bit RISC processor. The processor can also communicate via a USB link to the host PC. In this way the application PC can indirectly control all of the functions in the Smart Camera system. Due to transportation, the camera/projector setup could become misaligned. Therefore four 950nm infrared LEDs have been placed on the corners of the lower side of the canvas screen. The RISC processor can control these LEDs and their location determination. In this way the camera coordinates can automatically be aligned with the projector screen coordinates.

Prototype Implementation

The Smart Camera SoC architecture has been designed using Verilog and implemented on an Altera Cyclone II EP2C70 FPGA.

A 5 mega pixel (2592x1944) digital camera is used with an infrared sensitive lens and 950nm infrared bandpass filter. The camera can be programmed in resolution and field of view. The camera has on-chip 12-bit ADC and is used in our application at its maximum parallel pixel output rate of 96 MHz. The frame rate is determined by this maximum output speed, by the resolution chosen and by the shutter width. The full resolution frame rate is 15 frames/sec. At 640x480 VGA resolution frame rates of 150 frames/sec are possible. In our application we use a camera resolution of 1024x768 pixels for the brush image capture. This resolution and the field of view matches the resolution and the field of the projector. With this resolution a frame rate of 60 frames/second is used. There is a tradeoff between footprint resolution and frame rate. Higher resolutions and higher frame rates result in lower shutter times and weak lighting of the image sensor.

Using the image processing pipeline, described in the previous section, the brush position is determined by a real-time center-of-gravity calculation of the segmented footprint image. This is calculated immediately after the last pixel of a frame has been received. During the vertical blanking period of the camera, the footprint image around the center-of-gravity is retrieved from memory and sent to the application PC as UDP packets over the Ethernet connection.

The synthesis results with Quartus II 9.0 are shown in the Table I A 50 MHz NiosII/e processor is used as a 32 bit RISC processor for the overall control. For the Gaussian lowpass filtering power-of-two coefficients are used to economize on multipliers. Completely programmable coefficients would also be possible as in the current prototype architecture only 2% of the available multipliers are used (7 / 300).

The frame buffers for the storage of the current image and background image are implemented in ISSI DRAM memories. In case of controlled lighting environments both frame buffers can be left away.

User Experience

The user interface to the FluidPaint real-brush based painting system is very obvious and self explaining. As even young children are familiar to the water paint paradigm, they can use the FluidPaint system immediately (Fig. 11). A number of test persons, that never used the FluidPaint system before have been invited to make paintings. For each test 30 minutes were provided. Fig. 12 shows a painting made by artist Karel Robert. All test persons found the system to be very intuitive in its use.

Conclusions and further work

The Smart Camera SoC removes several frame period delays in the image transmission and processing chain for determining the brush footprint in the real-time real-brush FluidPaint system to a maximum of a single frame delay. It frees up the processing power on the host PC thereby drastically enhancing the response time of the overall system.

User interaction tests are planned in the future to evaluate the Smart Camera enhanced FluidPaint system and to get feedback on possible future improvements.

As the architecture has been designed in the Verilog HDL, it can be integrated together with an image sensor on the same CMOS chip as a Smart Camera System-on-Chip. Further research focuses on setting up a generic architecture for Smart Cameras in such a way that vision processors can easily be configured for other applications as well.

Fig. 1 .

 1 Fig. 1. Infrared brushes used in the IntuPaint system [12].

Fig. 2 .

 2 Fig. 2. IntuPaint system[START_REF] Vandoren | IntuPaint: Bridging the Gap Between Physical and Digital Painting[END_REF] in use.

Fig. 3 .

 3 Fig. 3. Operation principle of the FluidPaint digital painting system.

Fig. 4 .

 4 Fig. 4. FluidPaint global system setup Fig. 4. illustrates how the paint canvas is set up as a painting system. Two mirrors (M1 and M2) are used for folding the optical projection path of the projector. A hot mirror placed before the projector lens blocks the infrared light emitted by the projector. Fig. 5. is a photograph of the side view of the actual FluidPaint prototype. The bottom side of the paint canvas in use and mirror M2 is clearly visible.

Fig. 5 .Fig. 6 .

 56 Fig. 5. Photograph of the FluidPaint System Setup

  these waterfilms do not frustrate the internal IR light reflection. At the interface of the Layer 1 surface and the waterfilm the IR light leaves Layer 1 and propagates further inside the waterfilm under a similar angle. When it reaches the top of the waterfilm it is internally reflected again and propagates back into the transparent Layer 1.

Fig. 7 .

 7 Fig. 7. Interactive Painting in FluidPaint with real wet brushes.

Fig. 8 .

 8 Fig. 8. FluidPaint screenshot showing how the bristle structure of the brush influences the rendered painting result on the virtual canvas.

Fig. 10 .

 10 Fig. 10. Calibration LEDs for direct camera/projector alignment.

Fig. 11 .

 11 Fig. 11. Without any previous training, 4-year old Ann and 6-year old Ryan can use the water and real-brush based intuitive FluidPaint system in seconds.

Fig. 12 .

 12 Fig. 12. FluidPaint painting artwork made in 30 minutes.

  Di Fiore*, Frank Van Reeth*, Jing Liao$, Jinhui Yu$

	3.2	Smart Camera SoC Processor Architecture
										frame-buffer 1	frame-buffer 2
										DRAM	DRAM	Ethernet
	infrared Camera Sensor	Preprocessing	Lowpass filter	Background	Subtraction	Pop. Threshold	Contrast	Center of	Gravity	Histogram		Ethernet	Controller
				Programmable Interconnect Fabric
	infrared LEDs	Calibration Controlle r			32-bit Nios-II RISC processor			SoC Embedded Camera Processor
							USB
			Fig. 9. Smart Camera SoC Architecture

: Luc Claesen*, Peter Vandoren*, Tom Van Laerhoven*, Andy Motten

  *, Fabian Di Fiore*, Frank Van Reeth*, Jing Liao$, Jinhui Yu$

	TABLE I.	SYNTHESIS RESULTS OVERVIEW
	Description
	Total logic elements		12,867
	Total combinational functions	11,547
	Dedicated logic registers	4,891
	Total registers		5,012
	Embedded Multiplier 9-bit elements	7
	Total memory bits	724,548

Research funded in part by FWO, Research Foundation Flanders, project number G.A063.10

Luc Claesen*, Peter Vandoren*, Tom Van Laerhoven*, Andy Motten*, Fabian Di Fiore*, Frank Van Reeth*, Jing Liao$, Jinhui Yu$

Luc Claesen*, Peter Vandoren*, Tom Van Laerhoven*, Andy Motten*, Fabian Di Fiore*, Frank Van Reeth*, Jing Liao$, Jinhui Yu$


Oscar Alonso
email: oalonso@el.ub.es

Angel Diéguez

Control Electronics Integration toward Endoscopic Capsule Robot Performing Legged Locomotion and Illumination

Keywords: Active capsular endoscopy, gastrointestinal exploration, Brushless motor, liquid lens, control ASIC, electronics

Miniaturization of sensors and actuators up to the point of active features in endoscopic capsules, such as locomotion or surgery, is a challenge. VECTOR endoscopic capsule has been designed to be the first endoscopic capsule with active locomotion. It is equipped with mini-legs driven by Brushless DC (BLDC) micro motors. In addition it can be also equipped with some other sensors and actuators, like a liquid lens, that permits to enable advanced functions. Those modules are managed by an Application Specific Integrated Circuit (ASIC) specifically designed for the VECTOR capsule. The ASIC is a complete System-On-Chip (SoC) and integrates all the electronics needed to enable the legged locomotion and the sensing and actuating functions of the capsule in an unique chip. The SoC also permits other functions for endoscopic capsules such as drug delivery and biopsy. The size of the SoC is 5.1 mm x 5.2 mm in a 0.35 µm high voltage CMOS technology.

Introduction

The gastro intestinal (GI) tract, like any other part of the human body may be affected by infections or diseases. Because of the length of the GI tract and its location inside the human body, the detection and diagnosis of these diseases is very complicated. The method used by doctors when analyzing the GI tract is endoscopy. Basically, the endoscopy consists in introducing, inside the GI tract, a thin tube with a video camera and light at the beginning. The basic function of this system is to show to doctors the GI tract without any need of surgery. The tube can be inserted both orally and rectally. Endoscopy in the digestive tract is named differently depending on the area under study and the applied techniques, the procedures most widely used are:

 Esofagogastruoduodenoscopy (EGD), which includes the study of the oesophagus, stomach and duodenum. This procedure is used to discover the reason for swallowing difficulties, nausea, vomiting, reflux, bleeding, indigestion, abdominal pain, or chest pain. In particular, for this procedure the endoscope can be introduced through the nose or the mouth of the patients, who have been previously sedated in order to reduce the pain during the examination. In addition, the scope also blows air into the stomach in order to expand the folds of tissue and give the possibility of look through the folders.  Double Balloon Enteroscopy, which is used to study the small bowel. It is a newly developed endoscopic method which permits to explore the small bowel without any need of surgical intervention [1]. Using a simple push and pull method two balloons are used in this procedure to examine the small intestine in steps approximately 40 cm long.Double balloon endoscopy [2] is indicated in cases of unclear bleeding in the digestive tract, Chron's disease, unclear chronic diarrhoea, abdominal pain, intestinal polyposis, intestinal dysplasia and Celiac disease.  Push Enteroscope, which is also used for the study of the small bowel. During this procedure, a long, narrow, flexible gastrointestinal endoscope, known as a push enteroscope, is advanced into the upper gastrointestinal tract to examine and evaluate the proximal section (first one third) of the small bowel [START_REF] Muscarella | Infection Control, and the Clinical Practice of Push Enteroscopy[END_REF]. This procedure is indicated when the doctors are not able to identify the cause of obscure bleeding or some other GI disorder.  Colonoscopy, which is used to study the colon. The colonoscopy is the less invasive technique that uses a scope to examine the colon and the distal part of the small bowel. Colonoscopy has been available since the early 1970s and has become critical to the diagnosis and management of colorectal disorders 4].Although colonoscopy is a safe procedure, complications can sometimes occur. These include perforation a puncture of the colon walls, which could require surgical repair. Sedation during the examination, post-procedural abdominal pain and irritable bowel syndrome are the main drawbacks. However, with this procedure biopsies can be taken of any abnormal areas at the same time as the screening or diagnostic test is being done. Any polyps found can be removed during this procedure [4][START_REF]Factors determining post-colonoscopy abdominal pain: prospective study of screening colonoscopy in 1000 subjects[END_REF][START_REF] Park | Sedation-free colonoscopy using an upper endoscope is tolerable and effective in patients with low body mass index: a prospective randomized study[END_REF][START_REF] Rubin | Colonoscopic polypectomy: a critical review of recent literature[END_REF][START_REF] Levin | Complications of colonoscopy in an integrated health care delivery system[END_REF].  Flexible Sigmoidescopy (FS), which is similar to the colonoscopy. However, this procedure only examines the region from the rectum to the colon (also known as sigmoid). This procedure is one of the screening modalities for colorectal cancer. From the point of view of screening, FS clearly cannot completely exclude the presence of colon cancer in all asymptomatic people. However, flexible sigmoidoscopy every 5 years with or without fecal occult blood test (FOBT) is one of the screening methods recommended by major professional organizations. It identifies 50 to 70% of the advanced neoplasms, if any discovery of a distal neoplasia is followed up with a total examination of the colon by colonoscopy [9-10]. Endoscopy allows us to examine the digestive tract. Nevertheless, it also has some disadvantages. For example, it is a painful treatment for the patient because requires the introduction of a tube inside the human body (the scope has a diameter of 7-8 mm, however the pain associated with the procedure is not caused by the insertion of the scope but from inflating of the colon in order to do the inspection). Furthermore, endoscopy can not access the entire GI tract. For example, doctors are only able to study a small part of the small fine. Finally, endoscopy is not always effective, as the conditions are so adverse in the GI tract (long extension, folds of tissue, poor lighting) that the doctor is not always able to detect diseases, requiring a second observation or some alternative method.

The need to improve the existing method and the fact that the number of people around the world that access to endoscopy is about 15 million [START_REF]SMIT Congress 2005[END_REF] has led to the emergence of alternative methods like capsular endoscopy.

Capsular endoscopy is a minimally invasive procedure that permits to study the whole GI tract. Capsular endoscopy consists in a small capsule with the shape of a pill, which contains some electronic elements needed to perform the same task as the traditional endoscopy. In standard devices such electronic elements are basically a tiny camera, some LEDs for the illumination of the GI tract, a radiofrequency (RF) system needed to transmit the acquired images to the doctors and a battery (needed to supply the capsule) [START_REF] Glukhovsky | Wireless capsule endoscopy[END_REF][START_REF] Gheorghe | Olympus capsule endoscopy for small bowel examination[END_REF]. Figure 1 shows a picture of a capsular endoscope. The significant development in medical diagnostics and imaging has brought up a lot of new capsular endoscope products coming to the health care market. The capsular endoscopes have been able to minimize patient discomfort and pain during digestive tract screening with less risk of infection and harmless to body organs. This kind of medical procedure is less invasive and gives a great impact compared to the traditional method. As it occurs with classical endoscopes, capsular endoscopy also has some disadvantages. For example, being a portable device, the quality of the images and the frame rate is usually not the same as in traditional endoscopy. In addition, there is the possibility that the capsule suffers an obstruction in any part of the digestive system [START_REF] Boysen | Small Bowel Obstruction from Capsule Endoscopy[END_REF], and for this reason, capsule endoscopy is not recommended for those patients suspected of having a capsular obstruction, fistulas, motility disorders, patients with pacemarkers or other implanted electro-medical devices, dysphagia, and pregnancy [START_REF] Mishkin | Evaluation Report: wireless capsule endoscopy[END_REF][16].

The capsular endoscopes are known by different names such as wireless capsule camera, video pill, PillCam [START_REF] Glukhovsky | Wireless capsule endoscopy[END_REF][START_REF] Adler | Wireless Capsule Endoscopy[END_REF], EndoCapsule [START_REF] Gheorghe | Olympus capsule endoscopy for small bowel examination[END_REF] or Sayaka [18]. Their shape look similar to a pill or capsule and they are able to see areas, which traditional endoscopes are unable to see. Capsular endoscopes also gives extra convenience to the patient. After swallowing one of the aforementioned capsules, the patient may leave the hospital and return later after 8 hours. During this long period, the capsule captures images along the gastrointestinal (GI) tract while the patient is continuing the daily activity.

Although pill-shaped micro-cameras have existed for over 9 years by now, these systems are passive and are dependent to the peristaltic movement of the gastric wall to actively locomote. The camera takes thousands of pictures as it passes through the GI tract, but its position during this time cannot be controlled. Therefore, many research institutions around the world are exploring the possibility to have an active endoscopic device for medical inspection and therapy in the form of capsular robot [START_REF] Glass | A motorized anchoring mechanism for a tethered capsule robot using fibrillar adhesives for interventions in the esophagus[END_REF].

Compared to traditional endoscopy, capsular endoscopy performs similar operations in a less invasive way. In order to functionally emulate a traditional endoscope, a capsule endoscope must be equipped with 3 basic functions. These functions are:

 Vision system, which is in charge of illuminating the GI tract and acquiring images. The vision system is composed by at least one camera and some LEDs.  Communications system, which is in charge of sending the acquired images to the receiver placed outside the human body. It can be also in charge of receiving external orders.  Supply system, which is in charge of supplying the whole capsule. The supply system can be composed by batteries or by other systems like wireless power supplier. Although these basic functions are always needed, a capsule endoscope can be equipped with more functions (i.e. advanced functions) which will improve the diagnosis or add therapeutic possibilities. Figure 2 shows a scheme with the basic and advanced functions that can be added in a capsule endoscope and the components (sensors/actuators) required. Figure 2 describes also the electronics needed to enable each included function. The basic functions are those functions which are essential to build an operative capsular endoscope. Those functions have been described above. In addition, figure 2 also shows some of the most important advanced functions that can be added, as for example fluorescence, high frame rate, auto-focus/zoom, locomotion, drug delivery and biopsy. Basically, the fluorescent function permits a visible detection of early stage cancer by only adding some fluorescent LEDs to the capsule endoscope [START_REF] Baumgartner | A fluorescence imaging device for endoscopic detection of early stage cancer -instrumental and experimental studies[END_REF][START_REF]Real-time endoscopic fluorescence imaging for early cancer detection in the gastrointestinal tract[END_REF]. The high frame rate function permits to achieve more images of the GI tract and it also permits to perform the examination in real time. The autofocus/zoom functions requires the inclusion of a liquid lens (which is operated with high voltages), and it permits to acquire better images of the regions of interest. Finally, the drug delivery, biopsy and locomotion need special mechanism in order to perform their functions, for example it is needed a syringe or a pump for drug delivery, or a cutting mechanism for biopsy or some legs/vibration/magnetic system for active locomotion. The inclusion of new functionalities is accompanied almost always with the incorporation of additional electronics in the capsule endoscope. Therefore, the space constraints imposed by the capsule endoscope plays an important role because they limit the electronics that can be added. In addition, the packaging of the electronics is also important. For example, enabling each function with off-the-shelf components requires so much more volume than designing an ASIC with specific drivers.

The aim of this chapter is to present the architecture of an advanced capsule endoscope designed in the framework of the "Versatile Endoscopic Capsule for Gastrointestinal Tumour Recognition and Therapy" (VECTOR) FP6-European project. Such capsule must include some advanced functions in order to enhance capsular diagnosis and to enable therapeutic functions. The requirements envisioned for the VECTOR capsule are:

 Improving image analysis. In order to improve the diagnosis the system has to be capable of performing NBI or fluorescence analysis. In the VECTOR capsule these different analysis are achieved by using different LEDs in each case (e.g. for the Fluorescence function the capsule uses fluorescent LEDs).  Improving the vision system by enabling the autofocus and zoom function. This is achieved by adding a liquid lens.  High frame-rate: the state-of-the art capsules have a low frame rate. By increasing the frame rate the doctors are able to analyse more images. In order to achieve a high frame-rate it is necessary to implement a compressor between the camera and the transceiver.  Active locomotion. It permits to create space, stop and move forward and backwards the capsule inside the GI tract. The movement can be achieved by adding some legs to the VECTOR capsule or by using magnets.  Enabling therapeutic functions like drug delivery and/or biopsies. Implementation of the above functions with off-the-shelf components requires too much space. So, a specific solution in which a unique ASIC is used has been performed.

System architecture

An active capsular endoscope needs the intervention of a medical doctor during the exploration. Guiding of the capsule endoscope is necessary to explore in detail a region or to do therapy. This procedure depends on the acquired images and the criteria of the medical doctor, therefore the possibility of a human error have to be considered. Nevertheless, if some robotic capabilities are added into the capsule endoscope, the possibility of a human error can be highly reduced. For example, if the robotic capsular endoscope also processes the images searching for polyps or abnormalities. The robotic functions can be also very useful during navigation. Such robotic functions can be enabled by adding a central processing unit (CPU), like a microprocessor, into the capsule.

As the VECTOR project pursues the goal of improving capsular endoscopy by adding advanced functions, enabling therapy and robotic behaviour, the basic and essential blocks of the smart VECTOR capsule must be: powering unit, vision system, telemetry system, locomotion system and a CPU. The powering, vision and telemetry units are the basic functions for a passive capsular endoscope, however, for an active capsule it is needed to add the locomotion function. The CPU is required to enable the robotic behaviour of the capsular endoscope.

Due to the space limitations and the low power consumption requirements, it is preferable to use ICs. For this reason, the trend for this system architecture is to unify the CPU and the locomotive system in one ASIC (the control IC). With this system architecture the robotic behaviour is enabled by software.

Figure 3 illustrates the concept of the VECTOR capsule. The ASIC, the CMOS camera, the telemetry system and supply system are in charge of enabling the basic functions of an active capsular endoscope. The ASIC includes the drivers needed to enable advanced vision functions and locomotion. The selected actuator for enabling locomotion in the VECTOR capsule is a brushless (BLDC) micromotor. The most important feature of this actuator is that it can also be used to enable drug delivery and biopsy functions. Therefore, with the inclusion of a BLDC motor driver into the control ASIC it is possible to enable 3 different functions.

Figure 4 shows a representative schematic of the proposed architecture for the microrobotic VECTOR capsule. The next functions can be distinguished:  The Powering Unit, which is the most important module, since energy feeding is mandatory for all the other modules. A bottleneck affecting all the commercial capsules is the limited available battery power, providing typically 25mW for 6-8 hours [START_REF] Swain | The future of wireless capsule endoscopy[END_REF]. This amount of power is barely sufficient for low resolution images transmitted at low data-rates and certainly not enough for actuators and highly consuming modules. In the VECTOR capsule, a 3D inductive link is used to supply up to 400 mW (4). Details of the 3D inductive link can be found in [START_REF] Carta | Wireless power supply as enabling technology towards active locomotion in capsular endoscopy[END_REF][24]. In order to take full advantage from wireless power transfer, an energy storage system is also implemented, in order to provide high current peaks when needed.  The vision system, which is formed by a CMOS camera and 16 LEDs. The CMOS camera acquires images while the LEDs illuminate the GI tract. In particular, the camera is a monolithic 320×240 active-pixel RGB/gray level camera-on-a-chip sensor that has been developed by Neuricam [START_REF] Neuricam | [END_REF]. It has been fabricated using 0.18μm CMOS technology from UMC.

For the illumination, the solution is to use narrow bandwidth color LEDs switched on alternatively at high frequency to deliver a composite white light illumination. This solution has the great advantage to allow recording white images and chromatic images with a single illumination unit.  The transceiver: The endoscopic capsule needs a bidirectional data transmission for its correct operation, the endoscopic capsule has to receive orders from the medical doctors and it also has to be capable of sending the acquired data outside the human body. The downlink from the capsule to the outside world must be able to transmit a large amount of data. In addition, the available data rate defines the image quality of the endoscope. However, to get a high data rate one needs to increase the radio frequency of the carrier wave of the signal; but the higher the frequency, the higher the absorption of the waves by the human body. As the available power for the transmission is limited, a dedicated 2 Mbps FSK near-field transmitter has been designed and implemented to the endoscopic capsuled. The data error detection and correction are performed at the transceiver. Further details of the transmitter are given in [START_REF]Design of a 2 Mbps FSK near-field transmitter for wireless capsule endoscopy[END_REF].  The control IC, which has the control of the whole system. It controls the CMOS camera and the RF transceiver via the I2C bus. Taking into account the speed data transfers, an I2C bus has been selected to perform the interconnection of the different elements because it uses less wires (i.e. 2 wires), compared to other communication strategies like SPI bus (4 wires). In addition, the I2C bus is standard and it is widely used.

The control IC is also the one in charge of blinking the LEDs, driving 2 different motor drivers (used to enable locomotion, drug delivery and biopsy when it is needed) and driving a liquid lens (used for the autofocus and zoom functions).

As can be seen in figure 4, the ASIC does not control the compression system. Compression should be included in the CMOS camera or in the RF transceiver, however, as it will be described later, none of them include it. For this reason, in order to enable high frame rate it is needed to add an external element which has to be able to compress the acquired images and send it to the transceiver.

Control IC description

Looking at the basic features implemented on the commercial capsule endoscope, it is a big challenge to keep these inherent features of the capsule camera and integrate additional and advanced capabilities on a robot capsule, such as active locomotion and an auto-focusing. Especially if it is taken into account that it is needed a mixed signal ASIC in order to control and manage all these devices. Moreover, Such ASIC must be capable of working with low voltages and high voltages.

Beyond the trivial problems of integrating additional electronics on the same space, the difficulties arise to manage the actuators that enable the robotic functions. One of the key elements to solve such difficulties is to correctly select the technology to design the ASIC. For this reason, in this chapter the technology selection is detailed before entering in detail in the architecture of the ASIC and the design of the drivers.

Technology selection

The definition of the technology for the fabrication of the ASIC is a very important part in the design flow. Considering that only some technologies are available in Europe at low cost and that not all the features of the technology are available for low-volume users, the technology selection has to be done at the beginning of the design flow.

To fabricate the ASIC several technologies have been evaluated. It has been taken into account the integration density, the power consumption and the powering and driving voltages required for the capsule elements. All the technologies evaluated are available through Europractice Consortium or through Circuit Multi Projects (CMP) which offer low cost fabrication through Multi-Project-Wafer (MPW). On Table 1 there is a summary of technologies which can be used to design the IC for the VECTOR project. Deep-submicron technologies can not be used in VECTOR because the high-voltage requirements. The maximum voltage required for the liquid lens of the capsule is 60V, although the driving can be done at 50V. The high voltage technologies presented on Table 1 are designed to achieve voltages from 50V to 100V. Comparing these four technologies a higher integration is obtained with a lower characteristic size, which is obtained with AMIS I3T80 and AMS H35b4. Between these two possibilities it has been chosen Austria-Micro-System (AMS) H35b4 technology which can work with four metals, in front of the three metals on AMIS technology, giving the possibility to reduce the interconnections complexity and also the total area required for the system. Further reasons to choose the AMS technology is that EEPROMs can be available in the H35b4 technology.

Control IC architecture

All the drivers and control electronics of the VECTOR capsule have been embedded in the same IC fabricated with a 0.35 um HV CMOS technology except compression and error correction.

Figure 6 shows the architecture of the IC. The main block of the IC is the embedded 8051 microprocessor which is the control unit (8051 IP). The inclusion of the microprocessor into the endoscopic capsule it is needed because it gives more flexibility to the system and facilitates the debug. In addition, as the functions of the capsular endoscope are not known in detail, the addition of a finite state machine (FSM) is excluded. Finally, the microprocessor permits to add robotic functions to the capsule endoscope.

The microprocessor is equipped with 256 B of SRAM internal memory. It also has 2 kB SRAM of data memory and 8 kB SRAM of program memory. This numbers has been selected from previous experiences [START_REF]Guide to Design Ware DW[END_REF]. After the test of the ASIC it will be possible to exactly determine the amount of memory that is really needed. The memory type used is volatile. Therefore, each time the VECTOR capsule is poweredup, the program has to be uploaded in the program memory area. EEPROM memories have not been selected because they are not available by the technology provider in the case of MPW projects. The programming process is carried out by the Boot Loader (BL) that interprets and sends binary code received from serial port to particular program memory area. After the program is uploaded a Power On Reset is done to the processor and configuration registers. Figure 5 shows a simplified diagram of how the ASIC is programmed. Specific peripherals have been included for each of the functions of the capsule. The peripherals determine the dynamic power consumption of the capsule. This strategy allows to administrate the instantaneous power to not overpass the powering capabilities. The peripherals are the time stamp control unit (TSCU), the I2C control unit (I2CCU), the LEDs control unit (LCU), the lens control unit (LeCU), the clock generator unit (CGU), the communications control unit (CCU) and two locomotion control units (LoCU). Detailed description of the ASIC features is given next.

Time stamp control unit:

The TSCU is a simple FSM which is in charge of sending a synchronization bit to the transceiver. With this strategy the doctor is able to know when the images have been acquired.

I2C control unit:

The interconnections between the different elements are done using a standard serial bus I2C [START_REF]bit microcontroller family Datasheet[END_REF]. It is also possible to implement this bus with slight modifications.

Here are some of the features of the I2C bus:  Only two bus lines are required; a serial data line (SDA) and a serial clock line (SCL)  Each device connected to the bus is software addressable by a unique address and simple master/slave relationships exist at all times; masters can operate as master-transmitters or as master-receivers  It is a true multi-master bus including collision detection and arbitration to prevent data corruption if two or more masters simultaneously initiate data transfer  Serial, 8-bit oriented, bi-directional data transfers can be made at up to 100kbit/s in the Standard-mode, up to 400kbit/s in the Fast-mode, or up to 3.4Mbit/s in the High-speed mode.  On-chip filtering rejects spikes on the bus data line to preserve data integrity  The number of ICs that can be connected to the same bus is limited only by a maximum bus capacitance Figure 7 shows a schematic of the connections of the I2C bus.

Comunications control unit:

The CCU is in charge of receiving data from the telemetry system. It has to decode such data and send it to the microprocessor. The data is received at 4'8 kHz by a serial input. Therefore, the RX channel is used to send orders and information to the control ASIC. The communication link permits to control the capsule endoscope from outside.

The RX channel is usually in the sleep mode. To start the receiving process, the transmitter must send a logic one followed by the data (16 bits). Once received, the CCU generates an interruption (INT0) and it returns to the sleep mode. The interruption is processed by the microprocessor.

Clock generator unit:

The CGU is a simple digital block used to distribute the clock of the system. The CGU receives the external clock (10 MHz) which is used to generate the clock signals for the communication module, the LoCU, LeCU and LCU modules.

LEDs control unit:

The LCU is composed by a basic FSM which controls the current and the period to blink the LEDs and LED driver. The LEDs driver is composed of 8 transistors in parallel, with the Drain pins connected to the LED. Each transistor has a different W/L ratio. With this strategy, it is possible to control the current passes through the LED controlling the intensity of LEDs. Figure 8 shows the schematic of the LED driver.

Liquid lens control unit:

The liquid lens used in the capsule is the ARCTIC 416 liquid lens from Varioptic. It uses two different electrodes to apply the driving signal. This liquid lens can be modelled as a resistor with a capacitor in series between each electrode. The working principle is simple, the focal of the lens is changed when it is driven with a PWM signal of amplitude between 30 V and 50 V. The impedance of the lens depends on the frequency of the driving signal, and a PWM signal of 1 kHz has practical utility. Figure 9 shows how the focal of the lens is changed depending on the applied voltage (Vrms).

Although there are some ICs in the market which have been designed to drive the liquid lens [29-30], they consume too much area and, therefore, in the VECTOR capsule context, it is more suitable to design the liquid lens driver and implement it in the same ASIC where the other drivers are also placed.

The most typical circuit to drive an actuator like the liquid lens is an H-bridge driver. The main drawback is that the liquid lens requires high voltages (up to 50 V). Figure 10 shows the overall architecture of the liquid lens driver. The main element of the lens driver is the H-bridge (HB), which has two HV-PMOS transistors and two HV-NMOS transistors. The two outputs of the HB are connected directly to the liquid lens electrodes. The supply voltage of the lens driver (up to VDDH = 50 V) is generated by a DC-DC boost converter integrated partially (the inductor is external) into the SoC. The High Voltage level-shifters are needed to raise the driving signals to the operating gate voltages of the HV-PMOS transistors. Thus, the HV-PMOS transistors of the HB are driven with signal from VDDH to VDDH -2 VTHP (where VTHP is the threshold voltage of the HV-PMOS transistor). Under normal operation the driver focuses the liquid lens by changing the duty cycle of the boost converter control signal. The voltage supply is then changed between 30 V and 50 V. The control of the driver is performed by a FSM which is also included in the driver. A more detailed schematic of the boost converter is depicted in figure 11. As the technology used can afford 50 V, the boost converter also includes a small circuit to prevent the generation of a voltage greater than 50 V. Such circuit is composed by a resistor divider, a comparator and an AND gate. When the output voltage is greater than 50 V the output of the comparator supplies a logic zero to the AND gate, which automatically switches off the transistor M1. If the output voltage is lower than 50 V, the comparator supplies a logic one, and the control is performer normally by the FSM.

The M1 and M2 transistors are HV transistors. The M1 transistor is HV-NMOS transistor used as a switch. The M2 transistor is a HV-PMOS transistor used as a HV-Diode. Locomotion Control Unit: The micromotor to be used in the capsule endoscope is a micro BLDC motor SBL04 (4mm diameter) from Namiki. It is a very attractive solution for engineers working in micro technologies and medical field, servo lens and micropumps. The strong construction allows the operation also in extreme environmental conditions. In combination with planetary geared SPG04, 4 different reduction ratios, the micro motor is suitable for a wide range of applications [31].

The driver of the Namiki motor is based on a 3-phase structure. This driver is the most popular and simplest solution for the control of the BLDC motor. Figure 12 shows the driver composed by 6 MOSFETs and controlled by 6 different signals [START_REF]Proc. of the National Power and Energy Conference[END_REF]. Usually the MOSFETs are accompanied with a schottky diode in anti-parallel in order to manage the current generated in the inductance of the motor when one of the driving transistors is switched-off. Basically, the diodes maintain the transistors in the safe operating area (SOA). The better size reduction of the motor driver is achieved eliminating this external component. Despite its simplicity, it is needed to pay attention while designing the driver in order to minimize the gate voltage and the size of the transistors and, more important, to avoid freewheeling currents in the chip [START_REF][END_REF].

The motor works feed by current. When one phase has an input current, the second phase has an output current, and the third phase remains inactive. Alternating these phases the motor works properly. According to figure 12, in one stage the transistors 1 and 5 are switched on (a). Hence, the current flows through transistor 1 to transistor 5. In the second stage, transistor 5 is switched off and transistor 6 is switched on (b). In this case the current flows through transistor 1 to transistor 6. When going from (a) to (b), e.g., when transistor 5 is switched off, the parasitic inductance of the motor induces a freewheeling current that goes through the anti-parallel diode of transistor 2.

As commented, in order to reduce the area of the driver the diodes are not used. If no external diodes are used the freewheeling current must be driven by the parasitic anti-parallel diode of one of the MOSFETs. This implies a large quantity of current going into the substrate and, if possible, this current must be avoided or reduced. Therefore, the driving strategies used to avoid or reduce freewheeling currents in the design of a typical 3-phase driving structure with the h35 CMOS technology of AMS are:

 To implement a Synchronous Rectification (SR) with the controller [START_REF] Piscataway | Analysis of Electric Machines[END_REF][START_REF] Chapman | micromotors and electric drives[END_REF].  To control the induced freewheeling current by accurate control of the driving transistors [START_REF]Reducing torque ripple of brushless DC motor by varying input voltage[END_REF]. In this case, the control of the induced freewheeling current by an accurate control of the driving transistors was selected because the control is simpler than in the SR solution. Further details of this controlling method are given in [START_REF]Reducing torque ripple of brushless DC motor by varying input voltage[END_REF]. The charge pump is based on the Dickson-style charge pump [START_REF] Ahmadi | A new CMOS charge pump for low voltage applications[END_REF] (Figure 14). It is designed for low voltage applications and has low power consumption. To generate any supply signal from 0 to 5 V, the charge pump must be supplied with an external voltage between 0 and 1.8 V. The maximum power consumption is achieved when it is supplied by 1.8 V (an output of 5 V) and it is of 0.45 mW.

The driver is also equipped with a feedback stage to perform a sensorless control based on the measurement of the back electromotive force (BEMF) generated at the motor. The feedback stage is composed by three comparators and a R-2R digital to analog converter (DAC). Each comparator senses each motor phase. The DAC generates the voltage reference for the comparators.

Finally, in normal operation, the driver is programmed with a start-up sequence, and configured to achieve the maximum speed possible. The start-up sequence consists of aligning the rotor to a known position and soon afterward commutating it in slow speed. This start-up sequence is necessary to generate BEMF feedback from

Implementation results

The die photo is shown in figure 15a. The size of the SoC is 5.1 mm x 5.2 mm. The IC needs 2 externals capacitors of 36 nF and 300 nF, and an external inductor of 220 uH for its correct operation. The test of the ASIC has been performed using a wired VECTOR capsule prototype equipped with 8 mini-legs and 4 white LEDs. The procedure for testing was straightforward on an experimental board comprising one microcontroller and the ASIC. The microcontroller (PIC18F2550) is used to configure the ASIC each time we want to experiment with new programs. On the test bench, three connectors for BLDC motors and LEDs are connected to the ASIC. Further details of the test are given in [38][START_REF]Enabling multiple robotic functions in an endoscopic capsule for the entire gastrointestinal tract exploration[END_REF].

A graphical user interface (GUI) application written in JAVA language has been developed to communicate with the PIC18F2500 through USB port. Figure 15b shows the VECTOR capsule prototype connected to the experimental board. In the accompanying video it is shown how the capsule opens and closes the legs while the illumination is turned on and off simultaneously.

The wired capsule prototype allows us to measure the power consumption. Figure 16 presents the measured power consumption of each task performed by the VECTOR capsule prototype. The tasks are enabled/disabled by the microprocessor. The maximum power demand is in the LoCU module. Each BLDC motor is connected to 4 legs. Therefore, it is possible to move 4 or 8 legs. The power consumption of the motor during the start-up is higher than in the stationary. However, the start-up does not alter the behavior of the VECTOR capsule because it is done in a short time (i.e. 10 ms). Nevertheless, more exhaustive tests have been performed on the liquid lens driver and in the BLDC motor drivers in order to determine the correct operation of each block.

In the liquid lens driver the most problematic device is the boost converter, because if it fails it is required to use an external high voltage generator, which cannot be added in the capsule.

The DC-DC boost converter reported good results compared. An output of 50 V is achieved using a capacitive load at the output and 3.3 V at the input. Figure 17 shows the output voltage achieved using two different duty cycles (0.85 the red points and 0.7 the black ones) and different resistive loads connected at the output.

Although the liquid lens is not able to supply high voltages with low resistive loads, it is able to supply up to 50 V to the liquid lens. For this reason, the test of the full driver was prepared. The liquid lens has two electrodes in the external shape, for this reason it is needed to prepare a special board to connect the liquid lens in the right way. Figure 18 shows the boards used to connect the liquid lens The test consisted in generating an increasing voltage from 30 V to 50 V and applying the control signals generated by the driver to the liquid lens. With this strategy, it is possible to see how the focal of the liquid lens changes because the camera is acquiring images. Using this methodology, different images were acquired per different focals (i.e. different supply voltages). Figure 20 shows two pictures acquired by the camera using the liquid lens with a supply voltage of 30 V and 48 V. It can be seen that in the first image the liquid lens is not focalized while in the second image the liquid lens is focalized.

The test for the BLDC motor driver demonstrated that the driver can operate without the need of external diodes. This is a very important result because it reduces the area occupied by the BLDC motor driver, and the available space in the capsule endoscope can be used for other purposes. Figure 21 shows the motor signals during the start-up of the motor driver avoiding the freewheeling currents.

Conclusions

The next step needed to make capsular endoscopy the golden standard procedure in endoscopy consist in improving the diagnosis functions and enabling therapeutic functions. The solution proposed in this chapter consists in equipping a capsule endoscope with robotic functions. Such robotic functions can be enabled by equipping the capsule endoscope with more sensors and actuators, needed to increase the capabilities, and adding a control system, which will control such sensors and actuators. Instead of using multiple chips, the solution has been to concentrate all the control electronics in a unique chip which is able to generate high current and high voltages required for today actuators in microrobotics.

In addition, the ASIC can be programmed with an RTOS which permits to control the capsule robot in real time. This allows the capsule robot to act according to the input stimuli and devote more efforts to events which are more important.

The microrobotic solution improves the medical diagnosis because the doctors have a total control of the endoscopic robot. This provides better accuracy during the exploration because the capsule robot is able to approach and focus over the desired section of the GI tract. Furthermore, thanks to the active locomotion, it is possible to do a faster exploration, compared with existing solutions. It permits to exclude the heal area and a faster approaching to the diseased area. However, the microrobotic solution has still some problems due to space limitations and power consumption requirements.

Although miniaturization of sensors, actuators, control electronics, telemetry system, wireless power system, compression system, CMOS camera and illumination system is a fact, the inclusion of all these elements in an small capsule of reduced dimension is still a challenge. In particular, for the presented microrobotic solution, the size of the capsule robot is d 10mm and length 33 mm, which is bigger than the state of the art capsule endoscopes. Furthermore, the addition of so many elements to the capsule robot increases the power demand. Despite the fact that the wireless power system is capable of supplying 400 mW, it is not enough power for the capsule robot to perform more than one task simultaneously. Besides, the addition of the compression to the presented capsular endoscope introduces a problem because it cannot be integrated in the control IC due to the technology selection. It was finally implemented in a small FPGA (silicon blue FPGA), however the best solution is to integrate it in an IC specially devoted for this operation.

In conclusion, although the trend in capsular endoscopy is to equip the capsule endoscope with robotic functions and increase diagnostic and therapeutic capabilities, current limitations of space and power consumption makes this solution unfeasible at the moment. For this reason, new solutions have to be applied.

Fig. 1 .

 1 Fig. 1. Picture of a capsular endoscope.

Fig. 1 .

 1 Fig. 1. Basic scheme of the functions that can be added in a capsule endoscope. It also resumes the required electronics needed to enable such functions

Fig. 3 .

 3 Fig. 3. Basic scheme showing the VECTOR capsule concept.

Fig. 4 .

 4 Fig. 4. Schematic of the proposed architecture for the microrobotic VECTOR capsule.

Fig. 5 .

 5 Fig. 5. Simplified block diagram of the ASIC configuration during the wake up process.

Fig. 6 .

 6 Fig. 6. Architecture of the SoC.

Fig. 7 .

 7 Fig. 7. Connection of Standard-AND Fast-Mode Devices to the I2C-bus [28].

Fig. 8 .

 8 Fig. 8. Schematic of the LEDs driver.

Fig. 9 :

 9 Fig. 9: 1/f versus the applied voltage at the liquid lens.

Fig. 10 .

 10 Fig. 10. Overall architecture of the liquid lens driver.

Fig. 11 .

 11 Fig. 11. Schematic of the DC-DC boost converter. Outside the box are placed the external components.

Fig. 12 .

 12 Fig. 12. Scheme of the basic function of the motor in two consecutive steps.

Figure 13

 13 Figure13presents the architecture needed for the driving of the motors. The charge pump is in charge of generating up to 5 V and the level-shifter are in charge of adapting the controlling digital signals from the level 0 -3.3 V to the level 0 -5 V. Operating the transistors of the 3-phase inverter with 5 V gate voltage allows the use of smaller transistors compared with 3.3 V driving.The charge pump is based on the Dickson-style charge pump[START_REF] Ahmadi | A new CMOS charge pump for low voltage applications[END_REF] (Figure14). It is designed for low voltage applications and has low power consumption. To generate any supply signal from 0 to 5 V, the charge pump must be supplied with an external voltage between 0 and 1.8 V. The maximum power consumption is achieved when it is supplied by 1.8 V (an output of 5 V) and it is of 0.45 mW.The driver is also equipped with a feedback stage to perform a sensorless control based on the measurement of the back electromotive force (BEMF) generated at the motor. The feedback stage is composed by three comparators and a R-2R digital to analog converter (DAC). Each comparator senses each motor phase. The DAC generates the voltage reference for the comparators.Finally, in normal operation, the driver is programmed with a start-up sequence, and configured to achieve the maximum speed possible. The start-up sequence consists of aligning the rotor to a known position and soon afterward commutating it in slow speed. This start-up sequence is necessary to generate BEMF feedback from

  the motor coil, because BEMF feedback can only be detected when the rotor turns. If everything goes well and the BEMF reading shows an adequate value, the next commutation should be triggered by the incoming BEMF signals. The proper BEMF reading can be achieved through skipping one electrical commutation ahead by 120 degree and waiting for the zero crossing on the respective floating coil.

Fig. 13 .

 13 Fig. 13. Architecture of the motor driver.

Fig. 14 .

 14 Fig. 14. Schematic of the charge pump.

Fig. 15 .

 15 Fig. 15. a) Die photography. b) Image of the Test board and the Capsule prototype.

Fig. 16 .

 16 Fig.16. Measured power consumption.

Fig. 17 .

 17 Fig. 17. Output voltage of the DC-DC boost converter versus the resistive load connected at the output. Results obtained using a duty cycle of 0.85 (red points) and a duty cycle of 0.7 (black points).

Figure 19

 19 Figure19shows the test platform used for testing the complete system. It contains the liquid lens, a camera used to acquire images, the ASIC and the supplies sources needed to supply the ASIC. The liquid lens is in front of the camera, therefore it is

Fig. 18 .Fig. 19 .

 1819 Fig. 18. Board uses to mount and test the liquid lens

Fig. 20 .

 20 Fig. 20. Focalizing an image using the ASIC and the liquid lens. The first image uses a generated supply voltage of 30 V and the second image uses 48 V.

Fig. 21 .

 21 Fig. 21. Start-up of the improved driver using 12 steps to align the rotor.

Table 1 :

 1 High Voltage Technologies Available in Europractice and CMP

	Technology	L (um)	Price (€)	V (core)	V (I/O)	Runs/Year
	AMIS I2T100 0.7	600/mm2	5	5 / 100	5-EU
	AMIS I3T80	0.35	990/mm2	3.3	3.3 / 80	4-EU
	AMS CXZ	0.8	650/mm2	3.3 / 5	2.5 / 50	3-EU
	AMS H35b4	0.35	1000/mm2	3.3 / 5	20 / 50	4-EU/4-CMP