Amira Belhedi
email: amira.belhedi@mines-paristech.fr

Beatriz Marcotegui
email: beatriz.marcotegui@mines-paristech.fr

Adaptive scene-text binarization on images captured by smartphones

Keywords: Smartphone-captured images, adaptive binarization, scene-text, ultimate opening, area stability, Otsu algorithm, visually impaired people, assistive application

HAL is

Introduction

Smartphones are opening new possibilities for enhancing user's view of the world, providing applications such as geo-localization, augmented reality, etc.

This has become possible thanks to the high sensor resolution and computational power of these devices. In the framework of LINX project, we develop a smartphone application allowing visually impaired people to get access to textual information in their every-day life. A critical step for this project is to identify regions of interest in the images. One way to do this is to produce a binary image. However, image binarization in this context is hard: in addition to the absence of prior information on image content, acquired images can be of low quality. In fact, image acquisition conditions are not under control: taken by visually impaired people, several issues can arise such as blur, noise, bad lighting conditions, etc.

Several works have been devoted to finding a relevant and efficient binarization method. Some of them perform globally applying the same threshold to the whole image. One of the best known in literature is Otsu's algorithm [START_REF] Otsu | A Threshold Selection Method from Gray-level Histograms[END_REF]. Despite its performance on clean documents, it is not well suited to uneven illumination and to the presence of random noise. Other works perform locally, adapting the threshold to each image region. A popular method in literature is proposed by Niblack [START_REF] Niblack | An Introduction to Digital Image Processing[END_REF]. It is based on calculating a pixel-wise threshold by gliding a rectangular window over the gray level image. The threshold T for the center pixel of the window is defined as:

T = m + k.s, (1)
with m and s respectively the mean and the variance of the gray value in the window and k a negative constant. The main limitation of this method is the noise created in regions that do not contain any text, since a threshold is also applied in these regions. Sauvola [START_REF] Sauvola | Adaptive document binarization[END_REF] addresses this problem by normalizing the standard deviation by its dynamic range. This method outperforms the latter one, except in case of low-contrast text regions. A solution is proposed by Wolf [START_REF] Wolf | Text localization, enhancement and binarization in multimedia documents[END_REF] to overcome this drawback. The threshold formula is changed in order to normalize the contrast and the mean gray level of the image. More recent local methods have been proposed (e.g. [START_REF] Singh | A new local adaptive thresholding technique in binarization[END_REF][START_REF] Xiao | An efficient method of text localization in complicated background scenes[END_REF]) An interesting method that solves Sauvola's algorithm limitations is proposed by Lazzara [START_REF] Lazzara | Efficient multiscale Sauvola's binarization[END_REF]. It is a multiscale version of Sauvola's method. According to the authors, this method outperforms the previous ones, particularly in case of documents with text of various sizes.

However, it is not robust to very blurred text regions: they can be entirely removed or only partially detected with this method (this is demonstrated in section 4).

Local approaches give better results compared to global ones but often require more parameters to be tuned. The most difficult part is to find the optimal parameters' values for the set of images to be processed. However, adjusting those parameters with no prior knowledge of image content is difficult. In particular, adjusting the window size parameter with no information about the text size is not possible. the missing detection problem produced by existing methods in blurred and low-contrasted text regions.

Our approach is mainly based on a simple local Otsu's algorithm [START_REF] Otsu | A Threshold Selection Method from Gray-level Histograms[END_REF] performed on regions of interest automatically extracted from the image. Regions of interest are detected with ultimate opening (UO) [START_REF] Beucher | Numerical residues[END_REF] weighted by the area stability of regions. The UO is a residual morphological operator that detects regions with the highest contrast and has been used successfully for text detection [START_REF] Retornaz | Scene text localization based on the ultimate opening[END_REF]. In addition to grayscale information used by UO, the proposed method uses area stability information, derived from Maximally Stable Extremal Regions (MSER) method, to favor the detection of regions with a more stable area. The MSER [START_REF] Matas | Robust wide baseline stereo from maximally stable extremal regions[END_REF][START_REF] Matas | Robust wide-baseline stereo from maximally stable extremal regions[END_REF][START_REF] Kimmel | Are MSER features really interesting? Pattern Analysis and Machine Intelligence[END_REF] method is commonly used for scene text segmentation purposes [START_REF] Neumann | Real-time scene text localization and recognition[END_REF][START_REF] Zhang | Text extraction from natural scene image: A survey[END_REF][START_REF] Il | Scene text detection via connected component clustering and nontext filtering[END_REF][START_REF] Xu-Cheng Yin | Robust text detection in natural scene images[END_REF][START_REF] Sun | A robust approach for text detection from natural scene images[END_REF][START_REF] Qixiang | Text detection and recognition in imagery: A survey[END_REF]. It detects regions that are stable over a range of thresholds. It performs well most of the time but has problems on blurry images and characters with very low contrast [START_REF] Neumann | Real-time scene text localization and recognition[END_REF]. This constraint (area stability weight) is introduced in order to avoid the detection of regions with great changes in areas that are probably related to the merging of different characters or to the presence of noise.

Our binarization technique is the first step in the LINX processing chain.

Further steps consist in characterizing extracted regions and classifying them in text or non-text regions. Therefore, our objective is to maximize the recall (the number of detected characters).

Linx dataset and ground truth generation

Some public text databases with ground truth are available. However, most of them provide ground truth at the level of words (e.g. ICDAR'15 database [START_REF]ICDAR 2015, robust reading competition[END_REF]):

they are suitable to evaluate text localization, but not for text binarization.

Other databases with character-level ground truth exist, such as DIBCO [START_REF] Hdibco | [END_REF],

EPITA [START_REF] Lazzara | LRDE document binarization dataset[END_REF], IIIT 5K-word [START_REF] Mishra | The IIIT 5k-word dataset[END_REF], etc., but they only contain text documents.

In LINX project, we are not limited to text documents. Therefore, we had to produce our own annotated dataset (from LINX database).

LINX dataset

The dataset used is a subset of LINX database. It contains [START_REF] Xu-Cheng Yin | Robust text detection in natural scene images[END_REF]

Ground truth generation

In order to validate our method (described in the following section) and to compare it to existing methods, it is crucial to generate an accurate ground truth. This is a challenging problem, especially for the blurred regions of the dataset. For that, we used a semi-automatic approach. For each image of the dataset, a manually selected global threshold that maximizes the number of detected characters (i.e. maximizing the recall) is first applied and a binary image is generated. After that, all splitted or merged characters are respectively manually connected or disconnected, and all false detected regions are manually deleted. We note that, for blurred regions, the global thresholding does not work well. To solve that, different local thresholding methods [START_REF] Niblack | An Introduction to Digital Image Processing[END_REF][START_REF] Sauvola | Adaptive document binarization[END_REF][START_REF] Wolf | Text localization, enhancement and binarization in multimedia documents[END_REF] are performed on these regions and the best binarization result, for each region, is selected.

However, even after applying several local thresholding methods, some words are not correctly segmented into characters by any of them. We then chose to keep them as such and to add a flag (GT Level) in the ground truth file indicating the ground truth level (if GT Level = 1 it is a character-level, otherwise it is a word-level). Note that the overall procedure is applied on the image and on its inverse in order to detect both light and dark characters. Then results are merged in the same binary image and another flag (textP olarity) is added in the generated ground truth (if textP olarity = 1 it is light text, otherwise, it is a dark one). Finally, rectangular bounding boxes are computed from each connected component of the obtained binary images, since we have chosen a rectangle based validation (described in section 4). Even if a pixel-wise validation seems a better approach, we gave up the idea as it requires an accurate pixel-level ground truth that is extremely difficult to obtain in practice and may favor the method used for its generation.

The generated ground truth contains 3913 rectangular bounding boxes: 3253 characters and 660 words, 2216 bounding boxes with dark text and 1697 with light text. Some ground truth rectangles, cropped from our dataset, are shown in figure 2. LINX dataset and ground truth are available online1 .

Figure 2: Ground truth rectangles cropped from our dataset. It is a very challenging dataset that varies from very large and high-contrast text regions to very small, noisy and blurred text regions. Its annotation was a hard task, especially for low-contrasted or blurred regions. As explained in the text, all local binarization methods failed to segment some words into single letters. They were kept as such in the generated ground truth. Some examples are shown here.

Adaptive scene-text binarization

We propose a new adaptive scene-text binarization that does not require manual adjustment of the window size to the image content, and reduces the number of false alarms. The proposed method performs a local Otsu's algorithm on adaptive windows that are automatically extracted from the image. The adaptive windows extraction is mainly based on mathematical morphology.

In this section, we first describe the dataset pre-processing, then we detail the adaptive windows detection and finally we present the binarization method based on adaptive windows.

Pre-processing

The dataset used in this study is a set of smartphone images, usually color images. Given in the one hand that luminance contains essential information for binarization process, and on the other hand that we need a computationally efficient method, we start by converting color to grayscale images. For this purpose, we chose to use the following conversion formula:

Luma = 0.2126 × R + 0.7152 × G + 0.0722 × B.
A pre-processing filtering step is also required to reduce image noise. We chose to perform a bilateral filter [START_REF] Tomasi | Bilateral filtering for gray and color images[END_REF] of size 3 and σ gray empirically fixed to 20.

Adaptive windows detection

In general, the text is contrasted compared to the background (clear text on dark background or dark text on clear background), otherwise it cannot be read. The proposed method is based on this feature: it detects the high-contrast regions in the image and obviously, in its inverse, based on the ultimate opening operator (UO) introduced by Beucher [START_REF] Beucher | Numerical residues[END_REF]. The UO is a morphological operator based on numerical residues that detects the highest contrasted connected components in the image. The operator successively applies a series of openings γ i (opening of size i) with structuring elements of increasing sizes, i. Then, the residues between successive openings are computed: r i = γ i -γ i+1 and the maximum residue is kept for each pixel. Thus, this operator has two significant outputs for each pixel x: R(I) which gives the value of the maximal residue (contrast information), called the transformation in literature, and q(I) which indicates the size of the opening leading to this residue (the structure size that contains the considered pixel) called associated function:

R (I) = max(r i (I)) = max (γ i (I) -γ i+1 (I)) , ∀i ≥ 1 q(I) =    max {i + 1 | r i (I) = R(I)} if R(I) > 0 0 if R(I) = 0 (2)
The UO has been extended by Retornaz [START_REF] Retornaz | Ultimate Opening Implementation based on a flooding process[END_REF] to use an attribute opening [START_REF] Breen | Attribute openings, thinnings, and granulometries[END_REF] such as width, height, etc. The new definition of the transformation R and the associated function q are obtained by replacing, in equation ( 2), γ i by the considered attribute opening. In this case, the associated function q indicates information linked with the considered attribute. An example is shown in figure 3 to illustrate the intermediate steps of UO calculation. An opening of size 1 (γ 1 ) does not change the image, γ 2 removes one maxima and generates the first residue r 1 . An opening of size 3 (γ 3 ) removes larger regions and generates the second residue r 2 . At the end, γ 4 removes all regions and generates the residue r 3 . The last step of UO computation consists in generating the two resulting images: (i) the transformation R(I) and (j) the associated function q(I). For each pixel, the maximum residue r i is selected and recoded in R(I) and the size of the opening leading to this residue is recorded in q(I). For example, the maximum residue of the pixel located in the third line of the last column (= 4) was selected from r 1 and the opening size leading to r 1 is equal to 2. . Then, the outputs of the UO weighted by area stability function R α (I) and q α (I) are deduced (images (g) and (h)). Comparing figure 3(i) and figure 4(g), we can observe that R α contains less noise than R.

For our application, we chose to use the extension of the UO with the height attribute, since it is the most suited one for text detection. We set the largest opening considered equal to 1 3 of the image height, since characters are rarely larger. This choice is made in order to avoid artifacts that occur with a larger opening size. Very small regions (area < 15) are also not considered in order to avoid useless process. Note that the associated function q is not used in this study, we only use the transformation function R.

We chose to use this morphological operator for many reasons. First, it has the capacity of highlighting regions with the highest contrast which is suited for text detection. Then, it has the advantage to be a non parametric multi-scale operator and does not require any prior knowledge of image content. Finally, it can be performed in real time using the fast implementation based on image maxtree representation [START_REF] Fabrizio | Fast implementation of the ultimate opening[END_REF].

Despite its performance, this operator has the limitation to produce a lot of connected components, most of them do not correspond to characters. To reduce the number of false positives appearing with the UO, we propose to introduce a weighted function α i within the residue computation (equation 3), α i being the region area stability inspired from MSER method [START_REF] Matas | Robust wide baseline stereo from maximally stable extremal regions[END_REF] (used successfully as weighted function in [START_REF] Serna | Segmentation of elongated objects using attribute profiles and area stability: Application to melanocyte segmentation in engineered skin[END_REF]). The weighed residue r α i is defined as:

r α i = α i × r i , (3)
with α i the weighted function computed, for each pixel x, as follows:

α i = area i area i+1 , (4)
with area i the structure area (containing x) obtained from opening γ i . Using this weight function, the residue of connected component with low area stability is artificially reduced, compared to that with high area stability. The computation of the UO with area stability weight is illustrated on simulated data in figure 4.

An example on real images is shown in figure 5 on maxtree. In the tree, we only need to add to each node n, the area of its corresponding region and then, to weight in each UO iteration the computed residue with the node area stability value (see [START_REF] Fabrizio | Fast implementation of the ultimate opening[END_REF] for more details about tree based UO computation).

The transformation R α (area stability weighted) is thresholded with a global low threshold (set to 1) to remove regions with very low residues (low contrast or low area stability) leading to the image B α defined as follows:

B α (I) =      255 if R α (I) > 1, 0 otherwise
The binary image B α is used to extract the adaptive windows used in the next step of binarization. These adaptive windows correspond to the rectangular bounding boxes of B α connected components. An example is shown in figure 6 (c).

Binarization

A binarization process is applied to the original image on adaptive windows defined in previous section. In the simplest case, an adaptive window contains a character in its background. A simple Otsu thresholding process performs well in these cases. More complex windows may correspond to a set of characters gathered in their containing support. This may happen due to the fact that we have chosen a very low threshold (R α > 1) in order to ensure the detection of low-contrast characters. An example of this situation is shown in figure 7.

Applying an Otsu threshold on such adaptive windows does not detect the characters but the region containing them (see for example figure 7(d)). A multi-level Otsu algorithm, with three classes (one for the characters, a second one for the merged region and the third one for the background), is required in this case. In order to detect this merging situation we analyze R α in each connected component (CC) of B α , that we note R α CC . If the CC contains very different R α values and the most common value (the mode) is significantly lower than its maximum value, we assume that the CC has merged significant regions. Thus, a merging situation is declared if the following two conditions are satisfied:

• mode ≤ max 2 , with mode the value that appears most often in R α CC and max the maximum value of R α CC . If this condition holds, the CC contains regions with a contrast twice higher than the contrast of the largest part of it (the mode). This is the first hint indicating that significant regions are missed.

• modeP ercentage > 0.7, with modeP ercentage the percentage of pixels with R α value equal to mode. This condition confirms that the lowcontrasted region of the CC covers a significant part of it. This is in general the case when low contrasted region surround significant area.

An example of this process is illustrated in figure 7. The input is a crop from the original image corresponding to an adaptive window 7(a). The corresponding R α CC is shown in figure 7(b). If Otsu's algorithm is performed in this window, these characters will not be detected, as shown in figure 7(d). Analyzing R α CC , a merging situation is detected. The maximum value in R α CC is equal to 22 (max = 22) and characters are surrounded by a low-contrast region, its value in R α CC is equal to 8 (mode = 8). Thus, both merging conditions are satisfied. Then, the multi-level Otsu with three classes is performed in this adaptive window of the input image. The obtained result is shown in figure 7(e). The word "RENAULT" is well segmented.

We observe that character edges after performing the binarization step are cleaner than B α (figure 6(b)) and merged characters of B α are correctly segmented.

Post-processing

A post-processing is applied to the image obtained from the previous step in order to remove very small regions. We use a small area opening of size 15 (the image resolution is about 3200 × 2400). We show in figure 8

Validation

In this section, we validate the performance of the proposed method and compare it to the best known methods in the literature. The dataset used and its ground truth generation are detailed in section 2. In the following, the experimental protocol is first presented and the obtained results are then discussed.

Evaluation Protocol

The evaluation is performed comparing a list G of ground truth bounding boxes G i,i=1..|G| with a list D of binarized objects bounding boxes D j,j=1..|D| (with |G| and |D| the number of bounding boxes respectively in G and D). The rectangle based evaluation method presented by Wolf [START_REF] Wolf | Object count/Area Graphs for the Evaluation of Object Detection and Segmentation Algorithms[END_REF] is used. This choice is made for many reasons. First, it supports one-to-one, as well as one-to-many (splits) and many-to-one matches (merges). Then, the precision and recall measures are computed at the object level by imposing quality constraints (recall constraint t r and precision constraint t p ) to the matched rectangles. This gives a better estimation of false alarms and correct detections than the direct accumulation of rectangle overlaps. This is briefly explained in the following. The matching between G and D rectangles are determined according to conditions of different matching types (one-to-one, split and merge) based on t r and t p . Then for each G i , the recall value r i is defined as follows:

r i =            1
if G i matches against a single D j , 0 if G i does not match against any detected rectangle,

f sc (k) if G i matches against k detected rectangles (k > 1),
and for each D j , the precision value p j is defined as follows:

p j =            1 if D j matches against a single G i , 0 if D j
does not match against any ground truth rectangle,

f sc (k) if D j matches against k ground truth rectangles (k > 1),
with f sc (k) a parameter that controls the punishment amount. In our experiments, merges and splits are severely punished by setting

f sc (k) = 1 1+log(k)
which corresponds to the fragmentation metric introduced in [START_REF] Mariano | Performance Evaluation of Object Detection Algorithms[END_REF]. The recall constraint t r is set to 0.7 and the precision constraint t p to 0.4 (value recommended by Wolf). Obviously, splits in case of word-level annotation are not punished i.e. if G i matches against several detected rectangles and its ground truth flag GT level (introduced in section 2.2) is not equal to 1, then its recall r i is set to 1. Another A flag which is saved in the ground truth file is used for recall and precision computation: the textP olarity flag. As mentioned above, the same image can contain dark and clear texts. Then we perform each tested method twice, on the image and on its inverse. Then, for each G i , we compute the recall value from the suitable resulting image according to the textP olarity flag value and we select its(their) matching rectangle(s) from D, for precision computation. The false alarms that correspond to D j that do not match against any G i must be taken into account in precision computation. They can be selected from one of the two resulting images. In our experiments, we select them from the dominant polarity (based on the textP olarity flag). This choice seems appropriate and obviously does not influence the recall measure.

Results

For all methods presented in this section, the images are pre-and post-processed as described in sections 3.1 and 3.4.

We first verify that the use of adaptive windows based on the ultimate opening, extracted from B α , enhances substantially the results of local binarization methods. For that, three methods from the literature that use fixed window sizes (Niblack [START_REF] Niblack | An Introduction to Digital Image Processing[END_REF], Sauvola [START_REF] Sauvola | Adaptive document binarization[END_REF] and Wolf [START_REF] Wolf | Text localization, enhancement and binarization in multimedia documents[END_REF]) are tested on our dataset. Note that these methods are better adapted for document binarization purposes, even if they are currently cited in scene-text localisation approaches too. We use the implementation provided by Wolf [START_REF] Wolf | C++ code for document image binarization[END_REF]. For each of them, the optimal k value recommended by its author is used i.e. -0.2, 0.34 and 0.5 for respectively Niblack, Sauvola and Wolf methods. Concerning the window size value, we set it to 40 × 40 (default values of the distributed code [START_REF] Wolf | C++ code for document image binarization[END_REF]). Applying Niblack and Wolf methods on adaptive windows, B α , instead of fixed ones, enhances substantially the results and mainly the mean precision (table 1). It is increased from 8.4% to 50.2% in the case of Niblack's algorithm, and from 41.9% to 52.4% in case of Wolf's algorithm. The mean recall is also improved for both of them and exceeds 90% in case of Wolf's algorithm. However, we do not observe an improvement of Sauvola's algorithm with adaptive windows. This is probably due to its main limitation: the miss detection of low-contrast regions. fixed windows size adaptive windows R (%) P (%) F (%) R (%) P (%) F (%) We compare now our approach with three methods that do not require a window size adjustment to the image contents. The multiscale version of Sauvola's method presented by Lazzara [START_REF] Lazzara | Efficient multiscale Sauvola's binarization[END_REF], the morphological algorithm based on the toggle mapping operator TMMS [START_REF] Fabrizio | Text segmentation in natural scenes using toggle-mapping[END_REF] (ranked 2 nd out of 43 in DIBCO 2009 challenge [START_REF] Gatos | Icdar 2009 document image binarization contest (dibco 2009)[END_REF]) and the MSER method [START_REF] Matas | Robust wide baseline stereo from maximally stable extremal regions[END_REF] (the most cited for scene-text localisation [START_REF] Karatzas | Icdar 2015 competition on robust reading[END_REF]).Here, for TMMS and multiscale version of Sauvola's methods, the implementation provided by the respective authors with their recommended parameters is used. For multiscale Sauvola's algorithm, k = 0.34, the window size at scale 1 is w = 101 and the first subsampling ratio s = 3. For TMMS algorithm, the hysteresis thresholds cminL = 20, cminH = 45 and the thickness parameter p = 50. For MSER, the OpenCV implementation [START_REF] Opencv | [END_REF] is used with the parameters leading to the best results on our dataset: the ∆ value = 0, the minimum area = 15, the maximum variation between areas = 0. Our approach and TMMS lead to very similar f-measures (∼ 72%), followed by binarization on MSER-based adaptive windows (∼ 70%). Note that our binarization technique is the first step in the LINX processing chain. Further steps consist in characterizing extracted regions and classifying them in text or non-text regions. Therefore, our objective is to maximize the recall (the number of detected characters). In these conditions our approach is the best suitable solution for our application.

When observing the obtained results with more details, we see that for relatively good quality text regions, all methods give good results. An example is shown in the first row of figure 9. However, for degraded or low-contrasted text regions, we obtain better results with our approach. This is shown in the second and third rows of figure 9. Characters are removed or partially detected with TMMS and multiscale Sauvola's methods. With our approach we are able to detect even very low-contrasted, blurred and degraded text regions. However, it is not robust to shadows and high-saturated regions. An example is shown in the fourth row of figure 9: the word "TICKET CLIENT" is truncated by a shadow. It is not well segmented with our approach and multiscale Sauvola's method, TMMS method gives better results in that region.

We compare now the execution time of our method and the state of the art methods that give good results on our database (multiscale Sauvola's and TMMS methods). Given times do not include pre-processing and post-processing operations, only processing and I/O operations are included. The whole LINX dataset (16 images with a resolution of 3264 × 2448) is processed. For our method and the multiscale Sauvola's method, each image is processed twice (both polarities). Tests are carried out on a DELL PC with 3.4GHz (8 physical cores) Intel processor. Obtained results show that our method is slower (1.34s per image) compared to multiscale Sauvola's algorithm (0.86s per image) and TMMS algorithm (0.62s per image). There are also the post-processing operation that is time-consuming: about 0.22s per image and the pre-processing takes about 0.08s per image. Our code will be optimized in order to reduce the execution time and to be able to use it in real time. Second row, a blurred image. Multi-scale Sauvola approach misses most characters, TMMS misses part of them and our method and MSER based methods obtain better results . Third row, characters with very low contrast ("Da Vinci code") are detected by our method but not by TMMS, Sauvola and MSER based methods. Fourth row, an example of image with a shadow that leads to a missed detection of "TICKET" word with our approach, multi-scale Sauvola and MSER based methods while TMMS detects it correctly.

In conclusion, we can say that the proposed approach outperforms the methods from the state of the art on our challenging dataset.

Conclusion and perspectives

We have presented, in this paper, a new binarization method of text images acquired with smartphones. It has the advantage to be efficient, even on low 20 quality images, common in our application. In addition, it does not require any prior knowledge of image content, thus avoiding the parameters adjustment problem of local thresholding approaches (such as Niblack's and Wolf's) and giving good results. The proposed method is composed of two steps: 1) adaptive windows detection based on an original combination of the UO operator and the region area stability, 2) local binarization with Otsu's algorithm on these detected windows.

We have compared our method with several methods on a new challenging dataset introduced by us (containing 4000 characters). It is a subset of LINX images, acquired in real conditions, by visually impaired people. The heterogeneity of this database makes difficult the parameter tuning of popular local thresholding approaches.

The obtained results demonstrate: 1) the benefit of weighting the UO by the area stability function: the number of false alarms is significantly reduced

2) the efficiency of our method: more characters are detected compared to the tested methods, in particular in case of low-contrasted or blurred images, very common in our application and a good precision rate is obtained. The new database introduced in this paper and its ground truth are available online for research purposes.

Future work will be to optimize our algorithm in order to be able to use it in real time. It would also be interesting to improve its robustness to the presence of shadow and saturation regions, as it tends to detect contrasted shadows or saturated regions rather than the possible characters included in them.

  images acquired with smartphone cameras by visually impaired people. Some images are shown in figure 1. In spite of the reduced number of images, an important number of words is present: about 1200 words with 4000 characters. It varies from text documents to products exposed in a supermarket, very blurred text, noisy regions, shadow regions, high saturated regions, small and large texts, light and dark texts, etc.

Figure 1 :

 1 Figure 1: Some images from LINX dataset.

Figure 3 :

 3 Figure 3: UO computation step by step. The UO attribute used in this example is the height of the connected component. (a) Input image I. (b-e) results of height openings with size 1, 2, 3 and 4. (f-h) computed residues r 1 , r 2 and r 3 .An opening of size 1 (γ 1 ) does not change the image, γ 2 removes one maxima and generates the first residue r 1 . An opening of size 3 (γ 3 ) removes larger regions and generates the second residue r 2 . At the end, γ 4 removes all regions and generates the residue r 3 . The last step of UO computation consists in generating the two resulting images: (i) the transformation R(I) and (j) the associated function q(I). For each pixel, the maximum residue r i is selected and recoded in R(I) and the size of the opening leading to this residue is recorded in q(I). For example, the maximum residue of the pixel located in the third line of the last column (= 4) was selected from r 1 and the opening size leading to r 1 is equal to 2.

Figure 4 :

 4 Figure 4: UO weighted by area stability step by step computation . The same input image I of figure 3 is used here. First, the weighted functions (area stability of regions) α 1 , α 2 and α 3 are computed (images (a-c)). Then, they are used with their corresponding opening (figure 3 (b-e)) to generate weighted residues r α 1 , r α 2 and r α 3 (images(d-f)). Then, the outputs of the UO weighted by area stability function R α (I) and q α (I) are deduced (images (g) and (h)). Comparing figure 3(i) and figure4(g), we can observe that R α contains less noise than R.

  (c): R produces many spurious regions in the background (in red) that do not correspond to text regions. A thresholding of R can help removing these regions but may also remove lowcontrast text regions. The use of area stability weight avoids the detection of regions with important changes in area that are probably related to the presence of noise, artifacts and contrast variation in the background (figure 5(c)) or an unintended connection between components (see for example characters "INE" in figure 5(g)). Another example of the obtained transformation R α is shown in figure 5(d). The number of false alarms is considerably reduced. We also observe that characters are better separated with R α . An example is shown in the same figure. The transformation R connects 3 characters ("INE") as shown in figure 5(g), whereas R α disconnects them (figure 5(h)). Note that the area stability weight function is easily introduced with the UO implementation based

Figure 5 :

 5 Figure 5: Comparison between the transformation obtained (c)(g) with UO and (d)(h) with UO weighted by area stability function. These images are converted to color images for a better illustration (random colors are used to better illustrate the different residues values). The number of false alarms is reduced with area stability weight and characters are better segmented.

Figure 6 :

 6 Figure 6: An example of (a) R α > 1, (b) B α and (c) adaptive window: rectangular bounding boxes (in red) (results obtained on the image shown in figure 5(b)).

Figure 7 :

 7 Figure 7: Example of binarization merging problem illustrated on (a) a crop from the original image (figure 5(a)). This crop corresponds to an adaptive window defined by B α of figure (c) The word ("RENAULT") is contained in a low-contrast region. b) R α CC . c) B α d) Otsu algorithm applied to (a), all characters are missing. Analizing R α CC , the merging conditions are satisfied (mode = 8 ≤ max 2 = 22 2 and modeP ercentage = 0.8 > 0.7). (e) Multi-level Otsu with three classes is performed, all characters are correctly segmented.

  an example of the final result. Comparing this final result with B α image (figure 6(b)), we can state that the binarization step improves characters detection.

Figure 8 :

 8 Figure 8: Binary image obtained with our method (input image shown in figure 5(a)).

Figure 9 :

 9 Figure 9: Comparison with the multiscale version of Sauvola's, TMMS and Otsu performed on MSER adaptive window methods.The MSER is used for adaptive windows detection, then Otsu's algorithm is performed on these windows. First row, a good quality image. Good results are obtained with all methods. Second row, a blurred image. Multi-scale Sauvola approach misses most characters, TMMS misses part of them and our method and MSER based methods obtain better results . Third row, characters with very low contrast ("Da Vinci code") are detected by our method but not by TMMS, Sauvola and MSER based methods. Fourth row, an example of image with a shadow that leads to a missed detection of "TICKET" word with our approach, multi-scale Sauvola and MSER based methods while TMMS detects it correctly.

Table 1 :

 1 Mean recall (R) precision (P) and F-measure (F) comparison of fixed and adaptive windows binarization methods.

	Niblack	86.1	8.4	15.3	89.8	50.2	64.4
	Sauvola	65.4	36.9	47.2	62.1	37.3	46.7
	Wolf	67.7	41.9	51.7	93.0	52.4	67.0

http://cmm.mines-paristech.fr/Projects/LINX

Acknowledgments.

The work reported in this paper has been performed as part of Cap Digital Business Cluster LINX Project.