Balázs Sütő
email: suto.balazs@biotech.uni-obuda.hu

Zsolt Könnyű
email: konnyu.zsolt@biotech.uni-obuda.hu

Zsolt Tölgyesi
email: tolgyesi.zsolt@biotech.uni-obuda.hu

Tibor Skala
email: tibor.skala@gmail.com

Imre Rudas
email: rudas@uni-obuda.hu

Miklos Kozlovszky
email: kozlovszky.miklos@nik.uni-obuda.hu

3D Human Scanning Solution for Medical Measurements

Keywords: 3D reconstruction, RGBD, 3D human scanning

Today anthropometry can be performed with three-dimensional scanners. Our aim is to establish a low-cost, easy-to-use hardware and software solution, which is capable to do automatic, computer-based anthropometry and medical measurements for health care. We have designed and build a large, remote controlled turntable and a 3D scanner application, which can be used to digitalize 0.2-2,6 m tall real world objects into 3D models. With the turntable a 360° field of view can be reached even when using completely stationary, static sensors. Our software uses RGBD sensors for data acquisition, however it can be combined with other image sensors. With our solution prosthesis design can be more accurate and simplified, and we can provide for 3D modelers an efficient, real time method to scan and visualize human scale 3D objects. The scanned models can easily be used for rapid prototyping and 3D printing. The models can be exported into the most popular 3D modeling file formats for further analysis. Our solution decreases significantly the time, effort and cost of the clean up process of the 3D scanning. In this paper we provide information about our 3D scanning solution's design, and implementation, and we also describe its accuracy. The realized hardware and software solution provides a cheap and sufficiently reliable method to gather real time 3D depth and RGB data from human size objects for 3D reconstruction.

Introduction

Today anthropometrical data is used in a great variety of projects. Examples include designing [START_REF] Niu | Using Three-Dimensional (3D) Anthropometric Data in Design[END_REF], somatotyping [START_REF] Preedy | Somatotyping using 3D anthropometry: a cluster analysis[END_REF] and face measurement [START_REF] Loconsole | A New Marker-less 3D Kinect-based System for Facial Anthropometric Measurements[END_REF]. Traditionally data were collected using measure tapes and other measurement tools which was a physically exhausting task both for the subjects and for the staff. Since taking the measurements required physical contact with the subjects, obtaining data from many different geographical locations posed a problem and prolonged project time. With the appearance of 3D anthropometry these issues were solved but due to the additional hardware the cost of the projects increased significantly. Thanks to the popularity and evolution of RGBD sensors now they provide a cheap and reliable marker-less solution for 3D scanning. The scanned body models generated by 3D surface reconstruction based technologies can be used for anthropometrical data collection The goal of our project is the design and development of a 3D anthropometrical scanner system which can be used to scan human bodies for medical purposes. The proposed solution must have a low implementation cost. The scanner system must support dynamic and static camera settings, therefore scanning with a moving and a fixed camera must both be possible. Initial versions of the scanner application have to be compatible with the Kinect for Windows V1 and V2 sensors while later versions should also support other RGBD cameras. To create a technology independent software an application layer which hides the differences between the mentioned frameworks must be developed.

to Cloud-based Solutions

Data clouds can be used to store anthropometrical data and models collected by our software. By uploading the data to the cloud we are able to create a global database and share the data among a large group of users. A cloud-based database also enables the users to access the models and measurements without regard to their physical location.

The data cloud solves the problem of managing geographically distributed work. Due to the low priced nature of the solution, building huge networks with globally distributed measuring sites becomes a possibility.

Description of a Problem Solution

Our proposed solution for the problem is an anthropometrical scanner system based on a self-developed 3D body scanner application and an electronic hardware turntable.

Among the state of art algorithms we considered choosing the ICP algorithm due to its impressive speed, robustness and popularity. We selected two framework libraries which implement ICP, Kinect Fusion and the Point Cloud Library, because after some research they seemed to be the most suitable implementations to meet our goals.

The current version of our program uses the aforementioned frameworks, their data structures and algorithm implementations, but the whole process of 3D reconstruction can only be completed by using Kinect Fusion. The application builds a volumetric model from the RGB and depth data acquired from the sensor by integrating them into the model as sequence of frames. The model can be visualized in 2D using ray casting. Our program performs a 3D surface-reconstruction on the model, creating a triangle based polygon mesh, which can be exported into ply, stl or obj file formats. The RGB and depth frames can be saved with their associated camera pose and imported into Blender using a Blender script to create an UV mapped model.

Our system supports dynamic and static camera settings. To create a technology independent software we developed an application layer that hides the differences between the mentioned frameworks.

To automatize the scanning process we created an Arduino based scan turntable. The turntable can be accessed and controlled from the application by a virtual COM port. By using the turntable, 360° scanning of objects becomes possible without actually moving the sensor around. Thanks to this possibility the scan process can be made more comfortable, faster and the number of possible errors due to manual scanning can be reduced.

It is also possible to use multiple stationary RGBD sensors for scanning [START_REF] Robinson | Estimating Anthropometry with Microsoft Kinect[END_REF], however this results in increased computation time and hardware costs. If this approach is used the sensors must be well placed around the subject so while providing the 360° FOV they would not interfere with each other. Compared to the turntable based method this solution is less mobile because if the scanning hardware are moved, the cameras always need to be recalibrated before scanning.

The application uses multiple threads, and is capable of vertex-color based color reconstruction. UV mapping and Infra camera support are currently in development.

In later versions we would like to make other RGBD sensors compatible with our software. The 360° angle of view could also be provided by using multiple sensors. The application itself consists of a modular backend and user friendly GUI. The backend supports the most popular 3D scanner software frameworks (Kinect Fusion [START_REF] Izadi | KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera[END_REF], Point Cloud Library [START_REF] Rusu | 3D is here: Point Cloud Library (PCL)[END_REF]) and a variety of 3D RGBD cameras. The aforementioned libraries provide the essential data types and algorithms for data acquisition and 3D surface reconstruction of 3D objects. Modularity means that in addition to being able to fine-tune the scanning settings, like the camera resolutions and properties of the 3D model, the data acquisition, data types, 3D modeling and reconstruction algorithms can also be changed if needed, to provide a highly customizable software. By using an additional HD camera it is also possible to obtain high definition RGB data for the scene which can be integrated into the 3D model by utilizing UV mapping to provide high definition textures. These HD textures can be stored with the exported surface-reconstructed body model and can later be used for a variety of medical purposes like automatic skin cancer detection. This technology is currently being implemented using Blender scripts.

The algorithm for projecting the HD texture on the 3D mesh:

1. Running a retopology script to normalize the mesh 2. Unwrapping of the mesh and generation of the UV map. This provides the coordinate mapping which transforms the acquired RGB values into the coordinate system of the mesh. 3. Iterating the camera poses (Each pose has an attached RGB frame.) 4. Using the 3D mesh to separate irrelevant data of the RGB frame, eroding it and applying a filter at the edge of the relevant area to prevent overlaps at the projecting process 5. Projecting the RGB data on the texture in the created UV system from every camera position.

Results

For testing we first measured certain body proportions of our test subjects like neck, hip and waist size by hand using a tape measure. After the scanning finished we exported the models from our software and measured the same proportions on the captured models using Meshlab. In the predefined scan space we managed to scan people in relatively good quality using a low-end desktop PC. By utilizing the maximum scanning precision and resolution of the Kinect sensor and choosing the correct parameters for the scan the errors of the scan were under 0.8 cm-s in all cases.

During testing we used a volumetric model constructed from 512 × 512 × 512 voxels and a model resolution of 256 voxels/meter. Using the same settings on a highend desktop PC we were able to process 30 depth frames/second allowing real-time reconstruction. We also tested different indoor lighting and camera setups, including stationary and dynamically moving Kinect sensors. While we found the reconstruction algorithms to be more robust by using a stationary sensor and the turntable, illumination only affected the quality of the RGB colors and captured textures, but not the quality of the mesh models.

Conclusions and Future Work

While the RGBD based technologies and methods cannot produce the same quality models as other approaches, they provide an affordable alternative. From the list of currently available RGBD hardware sensors we chose the two versions of the Microsoft Kinect sensor due to their high compatibility rating with other systems. Compared to other RGBD devices the Microsoft Kinect is relatively cheap and reliable. After normalization it can provide the necessary precision for 3D body scanning.

The Arduino [START_REF] Uno | [END_REF] based scan turntable in coordination with the well-placed RGBD sensors provide the 360° FOV which is necessary to scan the whole human body. The rotation speed and direction can be controlled from the scanner application. While it

Fig. 1 .

 1 Fig. 1. The architecture of our application solution.

Fig. 2 .

 2 Fig. 2. The Arduino based scan turntable designed and constructed at Biotech Laboratory for 3D body scanning.

Fig. 3 .

 3 Fig. 3. 3D scanned mesh model of man with a crutch.

is also possible to scan without using the turntable, in that case the comfort and reliability of the scan process is reduced. Currently our system is fully compatible ith the Kinect for Windows V1 and V2 sensors and uses a single camera for RGBD data acquisition, but we have plans for using additional cameras in the future.

During testing using the aforementioned devices and technologies we managed to scan 3D human bodies, and we found that errors of the scan were always below 0.8 cm-s. Color and colorless reconstruction were also tested. While minor movement errors were mostly mitigated by the ICP algorithm, generally the subject had to stand still during the whole scanning process. This was due to the limitations of the ICP and Kinect Fusion based real-time scanning approach. [START_REF] Izadi | KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera[END_REF] In the future intelligent marker management could be used to increase tracking and reconstruction precision and performance. We are also planning to extend sensor compatibility to other RGBD devices and making a variety of medical sensors compatible with our system to increase sensor modality.


Viktor Zoltan
email: viktor.jonas@3dhistech.com

Miklos Kozlovszky
email: kozlovszky.miklos@nik.uni-obuda.hu

Bela Molnar
email: bela.molnar@3dhistech.com

Semi-Automated Quantitative Validation Tool for Medical Image Processing Algorithm Development

Keywords: validation tool, automated validation, medical image processing

Cancer research and diagnostics is an important frontier to apply the power of computers. Researchers use image processing techniques for a few years now, but diagnostics only start to explore its possibilities. Pathologists specialized in this area usually diagnose by visual inspection, typically through a microscope, or more recently on a computer screen. They examine at tissue specimen or a sample consisting of a population cells extracted from it. The latter area is the area of cytometry that researchers started to support by creating image processing algorithms. The validation of an image processing approach like that is an expensive task both financially and time-wise. This paper aims to show a semi-automatized method to simplify this task, by reducing the amount of human interaction necessary.

Introduction

The project [START_REF] Jonas | Ploidy Analysis on Digital Slides[END_REF] we are currently participating in aims to reproduce a medical research and diagnostic method called ploidy analysis (PA) through image processing means. PA is a method to measure DNA content in cell nuclei as a basic cancer marker, and is considered as a segment of pathology, more closely image cytometry (ICM). In diagnostics PA is usually done by a machine called Flow Cytometer (and the family of assessments related to it flow cytometry (FCM)), an appliance that operates with a light beam directed at a transparent capillary, where objects are traveling in sheath fluid, facilitating laminar flow. The objects are measured through their optical properties like light scattering, but the result is more one-dimensional measurement functions of time. Image cytometry takes a different approach. Digital pathology is in the process of introduction into medical diagnostics. This new approach is based on the idea of taking traditional glass slide specimen to the computer screen through digitalization. This enables experts to use the monitor to evaluate the samples, and also use software tools to achieve the task. Expectations are that this approach enhances objectivity reproducibility and traceability of forming diagnoses. Image cytometry is the sub-field where the software tools are used to process the digitized sample, enabling its users to analyze vastly more objects than the traditional visualmanual method using microscopes for inspection and clickers for counting. The project we are working on aims to reproduce a flow cytometry analysis by image processing means [START_REF] Jonas | Nucleus detection on propidium iodide stained digital slides[END_REF] [START_REF] Jonas | Separation enhanced nucleus detection on propidium iodide stained digital slides[END_REF]. Is the ICM approach viable as a diagnostic approach? This is a twofold question in itself: is it possible, and is it capable of sufficient throughput (equivalent of FCM)? The hypothesis of course is that it is possible, but measurements have to be taken to confirm.

A crucial part in applying (image processing) algorithms in forming medical diagnoses, is validation. There are two possible routes: validation by comparison to a validated approach (in this case FCM result), or the method used for validating FCM in the first place: clinical a study. The clinical study approach was chosen to eliminate the inherent error accumulation, and the simpler procedure regarding laboratory access and overall expenses in human work. This approach is semi-automatic in the sense of needing manual input for the quantitative validation in the form of the planar coordinates of the reference objects. The novelty in our approach is to decrease the human interaction as much as possible. To achieve this validation process was separated into two steps, only the first needing direct expert interaction: marking the objects to detect on the sample. Something similar was done traditionally: using a clicker counter, while examining the sample with a microscope. This we named the quantitative part. It is possible to add a qualitative part that relies on the result of a quantitatively sound detection as reference, for measuring object detection quality.

This paper aims to present an algorithmic tool for first approach validation of the image cytometry algorithm described, and possibly other image segmentation projects. The tool is constructed to be also useful in image processing algorithm development, by enabling continuous comparison to results of pervious variants.

Benefits from Cloud-based Engineering Systems

The image processing of medical images regarding small objects usually entail tiled image processing. This, in itself is a problem that scales well in parallel processing. Comparing or validating the results of an above mentioned algorithm can also be organized for highly parallel processing. More importantly considering the amount of samples to analyze in a pathology lab (that are usually highly centralized facilities), that should be done more quickly, than manual/visual inspection of the sample, to have relevance in diagnostics. A glass slide has the useful area of roughly 15x25mm. The optics in the hardware our project used for sample digitization enables ~0.2µm/pixel resolution. This means a bit more than 9GB of image data for each image channel. (Naturally for storage it can be compressed, but processing usually uses the uncompressed, full magnification data.) Though our project uses single channel images, but a medium sized diagnostic lab works with a few hundreds of these samples a day [START_REF] Stathonikos | Going fully digital: Perspective of a Dutch academic pathology lab[END_REF].

Recent papers discuss the role of cloud in medical image processing like [START_REF] Kagadis | Cloud computing in medical imaging[END_REF]. Considering the amount of data accumulated in digitized glass slides the storage advantages of clouds seems also an option to explore. Creating the required storage capacity, the security issues and the problem of maintaining these may be easily solved through cloud. Taking a step back another dimension of clouds can also be valuable for this area and that is the consultation over medical cases, the possibility to easily share and inspect samples, or even cases (sets of samples corresponding to a patient) are already feasible, but with pre-cloud techniques his can be solved only in a less than ideal manner.

Related Literature

There is extensive literature on the evaluation of image segmentation quality and validation. We started with an earlier work in mind; that work was a case study of a clinical validation [START_REF] Krecsak | Technical note on the validation of a semi-automated image analysis software application for estrogen and progesterone receptor detection in breast cancer[END_REF]. The categorizing approach of [START_REF] Zhang | Image segmentation evaluation: A survey of unsupervised methods[END_REF] was of great help to widen our field of view in this area. And other works like [START_REF] Ledig | The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)[END_REF], which used the same supervised approach as we planned, were of great help to select the depth and scope of this paper.

Discussion

Image processing projects usually encounter at least two types of demand considering validation/comparison. One is the actual validation of the segmentation algorithm or the software solution, where segmentation result usually has to be compared to human "segmentation" results (as in a clinical study -supervised approach). The other case is during the development of the algorithm that compares the algorithm to its previous version or to a completely different approach to confirm or measure the change or difference, to be able to rank them, and improve the best one further. Accordingly the proposed software application is constructed to compare two measurements. This enables us to consider it as an ordering relation of some sort between image processing algorithms and also as a validation tool, when comparing human validation as reference input. The analysis tries to find corresponding object pairs (or n-tuples) from both measurements (reference and the currently tested), compare them, and register their relation using result/error classes.

Data

A measurement to compare can be considered an array of records containing all measured morphometrical and colorimetrical properties of each detected object (cell nucleus in the case of the actual study). This paper focuses on reproducing the process of a manual validation project, only the morphology and location information is analyzed, though a simple check is conducted on other properties of the measured objects for enabling automatic discovery of unanticipated changes. Samples of human blood (lymphocyte nuclei) were used in this case. A 1 mm 2 area was analyzed on each of the 17 digital slides. The images processed were 0.1625µm/per pixel resolution compressed (jpeg 80). This means roughly 6150*6150 pixels on each sample.

Analysis

The simplest case is when object shapes (as polygons) match, and also all their measured (non-geometrical) parameters match. (For optimization purposes all shape comparison is preceded by the comparison of their bounding boxes, to ensure quicker analysis.) This group is labelled Perfect match. If the criterion for parameters fails the cluster is labelled Parameter mismatch. These two groups are most significant in synthetic tests, where objects with known geometry are segmented and analyzed. It can also be helpful during algorithm development to detect unwanted changes in segmentation or parameter measurement, as a step of automated testing protocol. This is why these two cases are handled separately, if at the end of this step all objects are accounted for, the analysis is over, if not, the remaining are passed to the next stage of the test. This next stage leverages the idea of neighborhood matrices and object overlap. Both measurements are assigned to a dimension of the matrix, and relations are recorded to the cell addressed by the two interacting detected objects. If their bounding boxes intersect, and also the actual shapes intersect the interaction is marked in the corresponding matrix cell as a binary flag (as visible in Fig. 2). Shape overlap is detected by simply rendering the two objects to the same image additionally, and counting overlapping pixels. Intersecting ratio is defined as the intersection area over the area of the object with greater area. When this matrix is populated a simple analysis is conducted. For simplicity name the horizontal dimension reference and the vertical the measurement. Also generate column and row sums for the matrix.

1:0 and 0:1 Object Relation

These cases can be found by collecting entries, where either the column sum or the row sum equals to zero. When there is a reference object that has no corresponding measurement object it is marked as false negative. The complementary case is labelled false positive (measurement for no reference). These objects are registered, and removed from the matrix.

1:1 Object Relation

Where the column and the row sum is also equals to one means that to the reference object only one measurement object is assigned, and also to that measurement object corresponds only (this) one reference object. These objects are further grouped into two classes. Where the centroid of the objects is on the same location the object pair is labelled as match, where the centroid locations differ, the label assigned is shifted. (In our case instead of centroid simply the center of the bounding box was used.) A shifted (or not perfect) match can be observed on Fig. 3.

This classification may later be used for a few purposes: When a measurement is run only on a smaller sub-area of the reference area, there is difference in coordinate systems of the two samples, but otherwise comparing two identical measurements. Similarly if the tissue sample is digitized twice, similar difference may be observed. A new digitization process may assign a new coordinate system for the digital sample, or the tilling can change the coordinate system locally, because of mechanical or parametric differences of the two attempts. Naturally these analyzed pairs of objects are also removed from the matrix.

1:n and n:1 Object Relation

In the remaining set of objects where the column sum equals one (and the row sum is greater than one, otherwise it would have been processed earlier) is the class of Merged, meaning more reference objects to a single measurement object. Conversely where only row sum equals one is labelled Split, assigned to the case where to one reference there are multiple measurement objects. All remaining cases are labelled as Residuum. (n:m relations are not desirable in this setting, found no purpose in further analyzing them.)

For the validation of the above described method the results of the previously published manual assays were used. To be able to compare these results the result classes of the two methods have to be arbitrated. The resulting classification results are visible in Table 1.

Table 1. Results of the manual and the automatic assay. Reference column designates the count of nuclei the expert marked. FP (false positive), FN (false negative), Match and Other columns contain the count of objects in the named cluster.

Human rating

Definition of the Evaluation Classes

Match: reference marker corresponds with exactly one measurement. In the proposed method the "Match" and "Shifted" classes, and also the half of the count of the "Split" class1 False positive: there is a measurement, but no corresponding reference object. Same in the proposed method and the other half of the "Split" class 1 False negative: to a reference there is no measurement object. Same in the proposed method Other: all other cases, usually where one measurement corresponds to more reference objects. Merged, Residuum classes The result classes of the assays can be considered categorical variables, the manual results as expected and the algorithmic results as measured values. The Chisquare goodness of fit test is used to confirm that the proportions the algorithm produced do not differ significantly from the ones the manual assay states.

The test was conducted using the significance level of 95% (p = 0.05). The column h in Table 2 contains whether the above null hypothesis stands, column pt contains the p-value needed to reject the null hypothesis at the actual significance level.

Conclusion

The results of the proposed comparator algorithm mostly concur with the results of the manual assay. In case of four samples the results seem to differ significantly. Further investigation is needed to uncover the cause in those cases. This may enable faster, more objective comparison of image segmentation. Being able to compare measurements in the magnitude of a few tens of thousands of objects automatically adds the possibility for some additional testing of similar image segmentation algorithms.

A few weaknesses were discovered. Comparison processing time grows quickly with the number of measured objects. Analysis speed is inherently at least O(n 2 ) because of the interaction-matrix. If the processing algorithm in itself is tile processing based (as in the case of our project), comparing the result subsets tile-wise seems a viable solution. In other cases by using spatial ordering of both the reference and the measurement is possible (like the storage or indexing the measurement in a quad tree manner), thus being able to construct interaction matrices for objects that possibly overlap in a distributed (and also quicker than O(n 2 )) manner.

Comparison of larger objects is not efficient or even not possible; rendering them on single images in memory may not be an option. Overlap calculation in their case should be implemented by an entirely different approach.

The separate step of detecting complete matches and geometric matches for testing purposes makes this twice as slow, the two "modes" should probably be used separately, mode chosen explicitly. To add a module with the possibility of supplying image masks and/or text files with a strictly set format as comparison input is also our future goal. This is necessary to be able to use the tool more generally or in other projects. Extending the comparison capabilities to the level of detail of the segmented objects' level is also a possibility.

Fig. 1 .

 1 Fig. 1. Sample area with segmentation mask. On the top-left a merged pair of nuclei is visible. On the bottom left area a few (very) low-intensity objects are located.

Fig. 2 .

 2 Fig. 2. Illustration of the relation matrix. Rows represent the items of the gold standard, columns contain the elements of the compared measurement. X-es designate objects where overlap is possible.

Fig. 3 .

 3 Fig. 3. Reference marker and detected object overlap test. Manually placed "reference" position marker on the left, segmented object mask on the right.

Fig. 4 .

 4 Fig. 4. A segmentation object containing two markers. Image processing "merged" the two nuclei into one segmented object.

Table 2 .

 2 The result of the Chi-square goodness of fit test of concordance between the human and the algorithmic rating. Sample ID designates the glass slide containing the cell nuclei; the other columns contain the results of the statistical test.

	sample ID	χ 2	pt	h sample ID	χ 2	pt	h
	1M01	1.7038	0.7900 1	1M13	5.1632	0.2710 1
	1M02	3.5843	0.4652 1	1M14	0.5417	0.9693 1
	1M03	21.0897 3.0e-04 0	1M15	12.1855 0.0160 0
	1M04	731.7800	0	0	1M16	0.4652	0.9768 1
	1M05	1.6605	0.7979 1	1M17	6.8198	0.1457 1
	1M06	1.4359	0.8379 1	1M18	3.5227	0.4744 1
	1M10	2.1032	0.7168 1	1M19	227.0211	0	0
	1M11	1.9999	0.7358 1	1M20	2.3150	0.6780 1
	1M12	0.3481	0.9865 1

V. Z. Jonas, M. Kozlovszky and B. Molnar

V. Z. Jonas, M. Kozlovszky and B. Molnar

In the manual assay the rule of thumb given to the expert was to mark the best fitting segmentation result as a Match all others as false positives.


Gábor Kiss
email: kiss.gabor@nik.uni-obuda.hu

Orsolya Eszter Cseri

Ádám Altsach
email: altsach.adam@nik.uni-obuda.hu

István Imre Bándi
email: bandi.istvan@nik.uni-obuda.hu

Levente Kovács
email: kovacs.levente@nik.uni-obuda.hu

Miklos Kozlovszky
email: kozlovszky.miklos@nik.uni-obuda.hu

High Resolution Digital Tissue Image Processing using Texture Image Databases

Keywords: DICOM, LBP, texture based image database, medical image processing, digital microscopy

Texture based image databases integrated with effective searching algorithms are useful solutions for many scientific and industrial purposes. Medical image processing of high resolution tissue images is one of the areas, where the cell/tissue classification can rely on such solutions. In this paper we are describing the design, development and usage of a specialized medical texture image database. Our primary aim with this texture database is to provide Digital Imaging and Communication in Medicine (DICOM) compatible texture image dataset for cell, gland and epithelium classification in histology. Our solution includes a Picture Archiving and Communication System (PACS) subsystem, which is mainly provide a communication interface (texture image searching and retrieval) and enables image processing algorithms to work more effectively on high resolution tissue slide images. In this paper we describe how our Local Binary Pattern (LBP) based algorithm benefits texture database usage when solving image processing problems in histology and histopathology.

Introduction

With the development of high resolution scanning and informatics, the practitioner's workflow has changed, and high resolution digital microscopy becomes possible. Slide sharing, in digital world is just easy as forwarding images via Internet, also visualization and archiving the images become simple tasks. Unfortunately, the migration into digital world did not solve all problems, and slide assessment tasks still remain complex and time consuming in the workflow. Practitioners -independently of whether they use traditional methods or new digital slides-, are sharing the same level of difficulties to do cancer grading and determine efficient treatments. In many countries (due to the large amount of samples) patients are waiting in long queues for their diagnosis result. Such waiting queues need to be avoided, because during waiting time, health problems may get worse.

State of the Art

We have done a market screening of the available high resolution digital microscopy image assessment software solutions. Our evaluated systems were the followings: the TissuemorphDP [1] image analyzer from Visiopharm, which is designed mainly for the population of nuclei examination, HER2-CONNECT [2] targeting tumor cells detection, Virtuoso, DeveloperXD [START_REF] Developerxd | [END_REF] and the MediaCybernetics [START_REF] Bickel | The earth mover's distance is the mallows distance: Some insights from statistics[END_REF] doing image of tissue analysis.

We have examined also some image databases namely: ASH Imagebank [START_REF] Ash Imagebank | hematology imagebank[END_REF], Image After [START_REF]ImageAfter texture database[END_REF], TinEye [START_REF][END_REF], RevIMG [START_REF]RevIMG reverse visual search[END_REF]. The ASH Imagebank is a hematology database with key word search. ImageAfter is a texture based approach with the string-based search method. Finally TinEye and RevIMG are Content Based Image Retrieval systems (CBIR). All the examined systems support only full size images instead of plain texture images.

MorphCheck is our medical image analysis platform working on high resolution digital microscopy images. It enables the load of high resolution tissue images, automatically detects the basic structures of tissue (nuclei, glands, surface epithelium) and provides quantitative and qualitative parameter measurements Recently we have added texture based image analysis to improve its object recognition accuracy. The users can do texture-base search and object recognition without manual selection.

Most of the image acquisition modalities are using the DICOM standard combined with PACS. We also have enabled the support of a DICOM/PACS based medical database system [START_REF]Digital imaging and communications in medicine[END_REF] in our system. DICOM and PACS technologies guarantee the standard storage for medical images and the software makes the additional extension of its abilities later. PACS server is being used by virtual servers in a cloud. We are using IaaS (Infrastructure as a Service), because we can scale to any amount of computer resources dynamically based on usage and other parameters. The scaling process is supported by the EC2 compatible cloud middleware.

The system measures a large number of morphological parameters on the detected image objects and store them in the database, all higher level analysis and decision is based on the archived database parameters. Image processing and classification tasks demand high computing capacity, thus the algorithms are able to run on various hardware architectures and support GPUs to enable higher performance [START_REF] Sergyán Sz | Parallel biomedical image processing with GPGPUs in cancer research[END_REF], [START_REF] Szénási | GPGPU-based data parallel region growing algorithm for cell nuclei detection[END_REF]. Our aim is to increase the efficiency of the software with new texture-based approaching.

Architecture

MorphCheck consists of two well separable parts. One of these parts is the plain algorithmic methods working on processing and detection. The other part is using the texture-based approach doing the image processing tasks (shown in Fig. 1.) The user may define workflows and freely use a combination of both parts of the image processing methods. A workflow contains tasks (e.g.: texture based and classifier algorithms) and can be arbitrary complex. The system also supports manual image annotation, where the pathologist makes the annotation of the tissue sample and uploads it into the database. The annotated images will be stored in the PACS system, and algorithms can reuse later on their data during classification processes.

Texture Based Search Algorithm

Our implemented LBP algorithm is based on the Local Binary Pattern [START_REF] Xiaoyuwang | An hog-lbp human detector with partial occlusion handling[END_REF] (shown in Fig. 2. and Eq.1). The value of LBP code of a pixel is given by: Fig. 2. part A shows a sample image with intensity pixels and the center pixel is being marked by blue color. Fig. 2. part B shows the differences between each pixels and the marked center pixel, and the differences are being written to the place of the original pixel. For example: The center pixel is 70 and the top left pixel is 47. The difference is -23, and it is written to the top left pixel place at the part B of the Figure 2. The last calculation is being based by Eq.1. Take each pixel on B image and if the current pixel greater than or equal 0, at C image the same pixel will be marked as 1. Otherwise it will be written 0. Finally the LBP code can be red by the clockwise direction. The task is to separate and classify the natural textures on the images. The input of the algorithm is the area where the examination happens, and those classes should be later identified on images (more pictures may belong to one class).

At first the Simple Linear Iterative Clustering (SLIC) algorithm slices images into pieces of texture [START_REF] Achanta | Slicsuperpixels[END_REF], [START_REF] Achanta | Slic superpixels compared to state-of-the-art superpixel methods[END_REF]. In the SLIC algorithm it is necessary to set two parameters: the size of the superpixel and how uniform it should be. The second parameter means how homogenous the colors should be in the superpixel. At high parameters we get inhomogeneous, compact and not random size superpixels (shown in Fig. 3.).

Fig. 3. A high resolution tissue image split into superpixels

After these steps we examine all superpixel in terms of belong in to a class. This process happens to the help of two parameters: Simple histogram and rotation invariant Local Binary Pattern histogram. With both parameters we calculate distance measure for all images. For the case of histogram we calculate the difference with the Earth Mover's Distance [START_REF] Bickel | The earth mover's distance is the mallows distance: Some insights from statistics[END_REF] metric. In the case of the LBP histogram we calculate the absolute difference of both histograms. Based on LBP to all superpixel we calculate a classification, after that we compare all superpixels classification with other. The robust classified superpixels are which superpixels classified same class, on the basis of both histograms (LBP and simple), all others are indeterminate superpixels. Currently, to avoid the false negative hits we classify them to the unclassified class, but later with the stable neighbors we will assign ranks to them.

Measurement and Test Environment

To measure image object classification efficiency, we have adapted some metrics from the literature. It is necessary to introduce some concepts to this:

• Reference result set: The pixels, which are marked in the reference image by a skilled expert.

• Test result set: The pixels, which are marked by the algorithms.

The selected pixel-based parameters from the literature are [START_REF] Yasnoff | Error measures for scene segmentation[END_REF]: • False negative (FN): Reference result set contains the current pixel, but the test result set does not.

• False positive (FP): The test result set contains the currently pixel, but the expert is not marked it on the reference result set.

• True positive (TP): The pixel is contained in both reference result set and test result set.

• True Negative (TN): Neither set contains the current pixel.

• Accuracy (TP+TN)/(TP+TN+FP+FN): This is the measure between 0% and 100%, where 100% means the reference result set and the test result set are equal. Thus the doctor and algorithms marked different pixels on the images.

• Recall (TP/(TP+FN)): This is a measure number indicates how much it found from the reference result set. If found all of them 1, else converge to zero.

• Precision (TP/(TP+FP)): Measurement of the hit. If the algorithm marked all pixels marked, the precision value is 1, else converge to zero.

Performance and Accuracy

In this section we show the performance of our LBP based algorithm, according to next aspects: runtime, accuracy, recall, precision. We use anonym tissue samples to do the efficiency measurements. To the tests we use 20 digital samples, which consist of 50 ROI (Region of Interest). From among these 32 ROI were healthy, 18 unhealthy. Table 1 contains all the data which has been produced by the tests.

Fig. 4. Result of accuracy examination

We run the examined algorithms under the tests, then we have calculated on all images the defined metrics (TP,TN,FP,FN). The accuracy is being shown by the Fig. 4. and Table 1. This metric of our implemented algorithm is in the range of 64.3-87,3% and the average is 75,99%.

The other important derived result is the so called "recall" (shown in Fig. 5.). Mixed results arise in the course of the tests. The LBP based algorithm tries to avoid the false positive hits. The area will be marked as undetermined if there is small confidence. This is the main reason of low recalls in the test result.

The third examined parameter was the precision (shown in Figure 6.). Our implemented algorithm is able to find pixels between 0,6 and 0,8 precision on all reference images. Among the examined 50 test, the founded pixels in 24 cases were correct, which means over 80% precision (which is very good if using such samples). Fig. 7 shows some test images. During visualization we are using some false colors by our algorithm: Yellow is a correct hit, black is undetermined area, blue is a background and green is a not examined tissue type. On the sample images there are some tissue structure: purple circle are glands, pink area is the epithelium. In this test we are searching for glands. In the B and D images the avoidance of the false positive hits showed by black color. Finally there are lots of good hits in a gland, epithelium and the background, among the black undetermined areas. The key of the algorithm's accuracy is the chosed distance measure technique. In the future we can probably enhance the accuracy by changing the current histogram based approach to the GLCM (Gray-Level Co-Occurance Matrix) based approach.

Summary

We have successfully implemented into the MorphCheck system an LBP-based texture classifier algorithm, which significantly provides more efficient object classification and segmentation then the previous solutions. After the integration we have done efficiency examination of our algorithm. We have used pre-defined metrics for the algorithm assessment. For the texture based algorithm we have combined MorpCheck with a DICOM/PACS based system and annotated a large number of tissue images. We used 1039 texture in PACS database and the LPB algorithm was working on these annotated textures. The new texture based algorithm and the combined PACS has been validated and facilitates diagnostic works significantly.

Fig. 1 .

 1 Fig. 1. High-level system architecture of MorphCheck

Fig. 2 .

 2 Fig. 2. Local Binary Pattern. A) Sample intensity pixels with center pixel, B) Differences from center pixel, C) LBP code in binary number and decimal system

Fig. 5 .Fig. 6 .

 56 Fig. 5. Result of recall examination

Fig. 7 .

 7 Fig. 7. A) Test image 1, B) Results of test image 1, C) Test image 2, D) Results of test image 2

Table 1 .

 1 Runtime, accuracy, recall and precision values of each image.

	Image	Runtime	Accuracy	Recall	Precision (0-1)
	1.	48032 ms	80,5561 %	34,989983 %	0,987352781
	2.	50503 ms	87,3210 %	74,7283407 %	0,627021984
	3.	45815 ms	78,4133 %	27,5994314 %	0,707845967
	4.	53523 ms	86,8289 %	86,2483716 %	0,69109668
	5.	48907 ms	82,3787 %	89,4566917 %	0,652158323
	6.	45962 ms	74,5288 %	60,1550501 %	0,98606324
	7.	49185 ms	71,5572 %	54,2035412 %	0,997744071
	8.	51919 ms	75,7369 %	68,6987809 %	0,984041932
	9.	47646 ms	83,8932 %	61,719451 %	0,968283442
	10.	46471 ms	84,8415 %	72,857387 %	0,626680672
	11.	46991 ms	70,9533 %	16,0547004 %	0,739163416
	12.	50978 ms	71,2926 %	18,0046956 %	0,621332388
	13.	56035 ms	71,4357 %	47,9457611 %	0,571567614
	14.	53736 ms	72,6450 %	89,063646 %	0,934838713
	15.	45531 ms	82,1188 %	50,8651431 %	0,883274726
	16.	51051 ms	85,9209 %	67,1163571 %	0,591183075
	17.	45935 ms	79,3216 %	83,922605 %	0,574389702
	18.	57948 ms	65,2986 %	66,5004848 %	0,956795831
	19.	50487 ms	71,7855 %	61,324101 %	0,646012997
	20.	46289 ms	64,3528 %	87,9682645 %	0,957433946
	21.	55556 ms	82,6466 %	59,9626072 %	0,632806935
	22.	42149 ms	78,7552 %	31,8393793 %	0,666676572
	23.	42207 ms	68,0102 %	49,1480765 %	0,896028264
	24.	53199 ms	78,2759 %	50,4916674 %	0,594485946
	25.	48327 ms	79,9055 %	23,9523506 %	0,986024626
	26.	54416 ms	74,5399 %	70,6376023 %	0,948759786
	27.	53046 ms	80,5891 %	78,4338479 %	0,722898764
	28.	46335 ms	64,9316 %	52,0194195 %	0,775494037
	29.	58761 ms	64,8349 %	36,1403908 %	0,705592692
	30.	52995 ms	64,8009 %	82,1711576 %	0,593973493
	31.	44834 ms	74,0871 %	26,2567997 %	0,824267399
	32.	42018 ms	78,9241 %	25,836577 %	0,819038487
	33.	49489 ms	81,2256 %	10,4580788 %	0,710334421

Acknowledgements. This work makes use some of the software results produced by the Hungarian National Technology Programme, A1, Life sciences, the "Development of integrated virtual microscopy technologies and reagents for diagnosing, therapeutical prediction and preventive screening of colon cancer "Hungarian National Technology Programme, A1, Life sciences, (3dhist08) project and the ÓE-RH 1104/2-2011 project. Authors would like to thank Semmelweis University and Major & Co. to provide us annotated tissue samples for processing and classification.


V Silva

M Barata

M A Vieira

P Louro

Light Memory Operation Based on a Double Pin SiC Device

Keywords: Optoelectronics, volatile memory, set reset logical functions

Experimental optoelectronic characterization of a p-i'(a-SiC:H)-n/pi(a-Si:H)-n heterostructure with low conductivity doped layers shows the feasibility of tailoring channel bandwidth and wavelength by optical bias through back and front side illumination. Front background enhances light-todark sensitivity of the long and medium wavelength range, and strongly quenches the others. Back violet background enhances the magnitude in short wavelength range and reduces the others. Experiments have three distinct programmed time slots: control, hibernation and data. Throughout the control time slot steady light wavelengths illuminate either or both sides of the device, followed by the hibernation without any background illumination. The third time slot allows a programmable sequence of different wavelengths with an impulse frequency of 6000Hz to shine upon the sensor. Results show that the control time slot illumination has an influence on the data time slot which is used as a volatile memory with the set, reset logical functions.

Introduction

A volatile memory is a state variable that only maintains its value over a period of time and then its value changes without any intervention on the system. A decisive change of the state variable, at any time, will behave with the same manner. The time during which the state variable value is recoverable and the number of times it can be read without changing its value define the efficiency of the memory. Different memory technologies can be combined to produce innovative memories like one that uses light to read the sate variable [START_REF] Guo | Nonvolatile memory based on the ferroelectric photovoltaic effect[END_REF], or molecules [START_REF] De Ruiter | Electrically Addressable Multistate Volatile Memory with Flip-Flop and Flip-Flap-Flop Logic Circuits on a Solid Support[END_REF].

The aim of this document is to answer the research question: how is it possible to build a memory using a pi'inpin device?

The hypothesis, based on previous research of the characteristics of the pi'inpin device [START_REF] Vieira | Double Pin Photodiodes with Two Optical Gate Connections for Light Triggering[END_REF], points out that charges are stored by illuminating the device in the front or back surfaces, and by using this characteristic it is possible to build a volatile memory. Experimental results have shown that different light sequences within a set of fixed tuples produce different outputs for the same tuples. This points out that there is a direct influence of the previous tuple upon the current one. Understanding this behavior will determine if the pi'npin presents itself as an effective volatile memory.

Contribution to Cloud-based Engineering Systems

The cloud, an icon used to represent the Internet, is now a service provided by many enterprises that host data space and offer applications that virtualizes storage and data bases. The Cloud is used in many circumstances has a common spot where to different users share data. It is a valuable asset for research as a means of connecting and sharing experimental data among researchers, so that validation can be performed, or to distribute tasks. Cloud services are "services delivered over the network" [START_REF]Software system engineering research group[END_REF].

Another appealing usage is the possibility that experimental setup can be linked into the Cloud, so that researchers use the same setup thus reducing costs and accessing instruments and devices that need skills which they do not necessarily have or need in their research. This would provide an out-of-the-lab research, especially to inaccessible areas or with hazard settings forbidden for researchers.

The experiments and device presented in this paper fits into that scenario; building an application that is hosted in the Cloud would connect an authorized researcher to our laboratory. The laboratory would have the necessary instruments switched on and controlled by a computer connected to the Cloud. The researcher would then set the inputs and expect the experimental results. Every experiment would the stored into a database in the Cloud where any researcher could see past experiments, comment results and suggest improvements.

Device Design, Characterization and Operation

The device consists of a 1cm 2 double pin/pin a-SiC:H photo detector with front and back optical bias shown in Fig. 1. The device operates within the visible range and input channels of several wavelengths (R: 626nm), green (G: 524 nm), blue (B: 470 nm) and violet (V: 400 nm) transmitted together as different bit sequences shine on the front of the device.

Fig. 1. Device configuration and operation

The combined optical signal is analyzed by reading out the photocurrent, under either front or back optical bias provided by constant illumination with (390 nm) LEDs [START_REF] Vieira | Double Pin Photodiodes with Two Optical Gate Connections for Light Triggering[END_REF].

In Fig. 2 the optical gain (α), defined as the ratio between the spectral photocurrent with and without applied optical bias, is displayed under front irradiation. The background intensity (φ) was changed between 5µ Wcm -2 and 3800 µWcm -2 . Results show that the optical gains have opposite behaviour under front and back irradiations. Under front irradiation and low flux, the gain is high in the infrared region, presents a well-defined peak at 750 nm and strongly quenches in the visible range, as shown in Fig. 2a). As the power intensity increases the peak shifts to the visible range and the spectral sensitivity can be deconvoluted into two peaks, one in the red range that slightly increases with the background intensity and another, in the green range, that strongly increases with it. In the blue range the gain is much lower. This shows the controlled long-pass filtering properties of the device. Depending on the background intensity selects the infrared or the visible spectral ranges; low fluxes select the near infrared region and cuts the visible one, the reddish part of the spectrum is selected at medium fluxes, and high fluxes tune the red/green ranges [START_REF] Vieira | Tuning optical a-SiC/a-Si active filters by UV bias light in the visible and infrared spectral ranges[END_REF]. Fig. 2b) shows the photocurrent gain under back irradiation. The background intensity (φ) was changed between 5µWcm -2 and 3800 µWcm -2 . Results show that under back irradiation, the gain is high in the blue region, presents a well-defined peak at 425 nm and strongly quenches above 500 nm [START_REF] Vieira | Near-UV background as a bridge between visible and infrared communication[END_REF].

Experimental Setup

The optical bias as a steady lighting is used in different applications of this device namely as a WDM communication element and digital light logical functions.

To use the device as a volatile memory the optical bias paradigm is changed and looked upon as The Front and Back Control signals use a 390 nm LED, set with an intensity current of 5 mA and 20 mA respectively. These values differ, so that the output of the pi'npin may be similar. The Data Sense signal is a single 626 nm LED channel pulsed at 6000Hz, set at an intensity current of 5 mA. These intensity current values were chosen so as to allow a clear visualization of the memory effect. The relative positioning of the LEDs and the pi'npin device can be visualized in Fig. 3.

A signal labeled as Dark is the photocurrent output when there are no Control signals and only the Data Sense signal is applied. It is used as a reference.

The double pi'npin sensor has a noise margin of 8nA [START_REF] Silva | Simple and complex logical functions in a SiC tandem device[END_REF].

Memory Effects

To introduce the memory theme a simple experiment is shown in Fig. 4a). On the top of the figure, there are the signals to guide the eyes. The experiment shown in Fig. 4a) is the result of the following sequential actions:

1-The Data Sense signal is applied and the pi'npin sensor's output is plotted and named Dark in Fig. 4a).

2-

The Back Control signal is applied during 1 ms, followed by complete darkness during 2 ms, and then the Data Sense signal is applied. The output is plotted and named Back in Fig. 4a).

3-The Front Control signal is applied during 1 ms, followed by complete darkness during 2 ms, and then the Data Sense signal is applied. The output is plotted and named Front in Fig. 4a).

These sequential actions were spaced in time between them by more than 2 minutes. This to ensure that one action does not have a big influence on the next one.

Observing Fig. 4 between 3 and 5 ms it is notable the reduced output of the Back in relation to the Dark, and the enhanced output of the Front in relation to the Dark. There is an influence of the Control signal over the Data Sense signal. Let the output be represented by the function f(control, sense).

To model this effect a digital component is presented in Fig. 4b). The digital component has all the input signals on the left hand side and the output signal on the right hand side.

The Control Phase input enables the Front Control and the Back Control signal, and the Data Phase input enables the Data Sense signal.

The operation of the digital component is better described with the aid of Table 1.

Table 1. Automata state variable table Line Inputs Internal State Output F B Ctrl S Dt Storeactual Storenext Q 1 F 0 1 - 0 - F - 2 0 B 1 - 0 - B - 3 - - 0 - 0 Storeactual Storeactual 0 4 - - 0 S 1 Storeactual Storeactual f(Storeactual, S) 5 - - 1 - 1 - - -
Presented in Table 1 are the automata transition states of the digital component. In this table the (-) symbol represents a don't care value, which means that whatever the value that the variable has on that state, it does not interfere on the subsequent action. When used as an output it means that the output in that case should not be considered. The (1) symbol means that the signal is active, and the (0) symbol that the signal is not active. The internal state is defined by variable Store, which is represented as Storeactual indicating the value that it holds in the present state. Storenext represents the value that Store will hold due to the input values. Line 1 indicates that when the Control phase is active and the input Front Control as a value F the next state will be set with that value. Line 2 indicates that when the Control phase is active and the input Back Control as a value B the next state will be set with that value. Line 1 and 2 also imply that the Front and Back Control are not set simultaneously. Line 3 shows that when there is neither a Control phase nor a Data phase the internal state variable will hold the same value; i.e. during the Hibernation phase the stores value does not change, and that the output is inactive. Line 4 indicates that when the Data phase is active the sensor output is a function of the present value of the internal state variable Store and of the Data Sense signal S; output = f(control, sense). Line 5 indicates that if the Control phase and Data phase are set simultaneously the component will have unpredictable results.

The three phases, Control, Hibernation and Data have to be studied for the understanding of the memory effect. The approach used will focus on the duration in time of the three phases. Some time setting will be fixed and others vary to better observe and understand the results.

Control Signal Phase Duration

This study uses Control signals with different time duration, from 1 to 5ms. After each Control signal, follows a fixed Hibernation phase that lasts 3ms after which a Data Sense signal is applied. Results show that there is almost no difference to the output produced by the applied Data Sense signal despite differences in the length of the applied Front Control signal. The Data Sense signal output is enhanced related to the Dark reference which is in conformance with the spectral gain presented in section 2. The results also show that higher duration of the Back Control signals reduces more the output of the applied signal when comparing it to the Dark reference. Comparing both Fig. 5 a) and b) on the Data Sense signal output, it is possible to infer the side on which a Control signal was set.

Constant Control Plus Hibernation Time

For visualization purposes the sum of the Control phase time plus the Hibernation phase time remains constant with a value of 3.1 ms. On successive experiments the focus will be on the Control signal duration, from 0.1 ms to 3 ms. The aim is to have the highest difference in photocurrent of the Data Sense signal output comparing the results of the applied Front Control or Back Control signal. The Data Sense signal is the same on all experiments, and is a random digital sequence.

Presented in Fig. 6a) is the experimental result of different Front Control signals followed by a short sequence of a Data Sense signal and in Fig. 6b) the experimental result of different Back Control signals followed by the same Data Sense signal. Results show that there is almost no difference to the output produced by the applied Data Sense signal despite differences in the length of the applied Front Control signal. The Data Sense signal output is enhanced related to the Dark reference which is in conformance with the spectral gain presented in section 2. Results also show that the output produced by the applied Data Sense signal reduces with the increase of the duration in time of the Back Control signal. The Data Sense signal output is quenched related to the Dark reference which is in conformance with the spectral gain presented in section 2. Observing both figures Fig. 6a) and b) it is clear that, the longer in time the Front and Back Control signals are the higher the photocurrent output difference between the Data Sense signals is.

Control Signal Current Intensity

The Control Signal is applied with different current intensities, 1mA to 30mA, followed by a fixed Hibernation time of 3ms, and then a Data Sense signal is applied.

Presented in Fig. 7a) is the result of each Front Control signal applied to the pi'npin and Fig. 7b) shows the results for the applied Back Control signals. The figures are truncated for the higher currents because its representation does not concern the result discussion, and would otherwise prevent a better visualization of the results. The results show that the higher the Front Control signal current intensity is, the higher is the output signal, when comparing to the Dark reference. Results also show that the output of the Data Sense signal is slightly reduced, when compared to the Dark signal, for increasing current intensities of the Back Control signal.

Observing both figures Fig. 7 a) and b), and regarding the Dark reference, it is better to use the highest current possible for the Back and Front Control signals due to the fact that they provide Data Sense signal outputs with greater differences to the Dark reference.

Conclusions

By comparing the amplitudes of the Dark signal output to Data Sense signal output it is possible to know if the Control signal beforehand was a Control Front or a Control Back signal. The Control Front and Control Back signals are pulsed ultra-violet wavelengths. The Data Sense signal is a 626 nm wavelength signal. The memory stores which of the control signal was used last, either the Control Front signal or the Control Back signal. The memory is read by the Data Sense signal indicating if the Control Front signal was used or if the Control Back signal was used. A volatile memory can be built using a single pi'npin device.

Fig. 2 .

 2 Fig. 2. Spectral gain (αB) at λ=390 nm under a) front and b) back irradiation

  two light Control signals; the Front Control and the Back Control. The operation has three phases, the Control, the Hibernation and the Data phase. During the Control phase only the Front Control and the Back Control signals may be presented to the sensor. The Hibernation phase has no illumination signals; the sensor is in complete darkness. During the Data phase the sensor is subjected to the Data Sense signal.

Fig. 3 .

 3 Fig. 3. LEDs, signals and the pi'npin device.

Fig. 4 .

 4 Photocurrent[µA]

Fig. 5 .

 5 Photocurrent[µA]

Fig. 6 .

 6 Photocurrent[µA]

Fig. 7 .

 7 Photocurrent[µA]

Acknowledgments

. This work was supported by FCT (CTS multi annual funding) through PIDDAC Program funds and PTDC/EEA-ELC/111854/2009 and PTDC/EEA-ELC/120539/2010.


H I Pousinho
email: hpousinho@gmail.com

P Freire

J Esteves

V F Mendes
email: vfmendes@deea.isel.pt

C Pereira Cabrita

M Collares-Pereira
email: collarespereira@uevora.pt

H M I Pousinho

V M F Mendes

Optimal Generation Scheduling of Wind-CSP Systems in Day-Ahead Electricity Markets

Keywords: Concentrated solar power, wind power, mixed-integer linear programming, transmission constraints

des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

Introduction

Renewable energy grid integration increased in the E.U. to fulfill the Energy -2020 initiative. Energy efficiency improvement, renewable technologies and lower emissions are priorities to ensure environmental sustainability [START_REF]Energy 2020 -A strategy for competitive, sustainable and secure energy[END_REF]. Wind power systems and concentrated solar power (CSP) systems are engaged in the meeting of demand increase in the Southern Mediterranean, profiting from favorable renewable energy sources. But, the non-controllable variability and limited predictability of the energy source of these systems [START_REF] Taylor | Competitive energy storage in the presence of renewables[END_REF] conveys real-time uncertainty posing capacity planning challenges related to the tuning of ancillary services needed to ensure system stability and reliability [START_REF] Lannoye | Evaluation of power system flexibility[END_REF]. Also, real-time uncertainty on renewable energy availability can increase costs if an unfitting prediction occurs. Hence, a coordination approach seems to be an adequate required option in order to reduce the uncertainty in power output. The wind-CSP systems coordination is a reliable option, enabling to accommodate energy integration into the power grid and day-ahead electricity markets [START_REF] Sioshansi | The value of concentrating solar power and thermal energy storage[END_REF], as well as to increment the dispatchability attribute on the accommodation. The non-dispatchable characteristic of CSP systems is lessened with thermal energy storage (TES) systems, enabling plant base load support. TES not only allows offsetting generation deficit during noninsolation periods, reducing real-time net generation variability, but also allows shifting of generation towards appropriate periods [START_REF] Dominguez | Optimal offering strategy for a concentrating solar power plant[END_REF]. TES has proved to be a reliable option [START_REF] Madaeni | How thermal energy storage enhances the economic viability of concentrating solar power[END_REF], allowing control on the energy, improving the capacity value and profits. Moreover, TES further increments the dispatchability and can reduce the running of conventional thermal power plants in short-term ancillary services [START_REF] García-González | Stochastic joint optimization of wind generation and pumped-storage units in an electricity market[END_REF], reducing the need for spinning, implying a noted reduction in reserve costs. Also, TES allows reducing high marginal cost units when an unpredicted drop in renewable energy input occurs. Therefore, the line of research of coordination approach for wind-CSP having TES systems is regarded as an important promising one offering new technical challenges to the researchers and this paper presents a contribution on this regard.

The paper presents a mixed-integer linear programming (MILP) model for solving the generation scheduling problem of wind-CSP having TES systems. The aim is to achieve the optimal generation schedule that maximizes the profit of a power producer taking part in a day-ahead electricity market. The optimization approach considers not only transmission line constraints, but also technical operating constraints on wind power systems and on CSP systems to enable a true management support for the integration of renewable energy sources in day-ahead electricity markets.

Relationship to Cloud-based Solutions

Cloud-based solutions can help the processing of approaches for helping trading in a day-ahead electricity market in order to take greater advantages of bids, requiring methodological approaches demanding computational effort not currently available to all power producers. Among these approaches the ones for the solution of the problems concerned with scheduling: energy management, unit commitment and energy offers are particular vital for upholding power producers. The approaches to solve these problems have been limited by the available computational resources, i.e., details concerning some reality are disregard in view of the excessive use of processing requirements. These limitations are evident for large systems and lead to solutions not addressed in convenient agreement with the reality, because the computational resources do not allow the usage of data or modeling for delivering more real adequate solutions.

Cloud-based solutions are able to contribute for upholding a power producer exploiting wind-CSP systems by setting out an adequate availability of computational resources needed to address scheduling problems in due time. Although of the uncertainty on the wind and solar power, i.e., intermittency and variability, cloud-based engineering systems will deal with problems having a better approach to the full reality in due time with gains in terms of profit. Cloud-based solutions will help on the smoothing of the electricity price due to the fact of having bids of wind power with higher level of energy and with more certainty in what regards the satisfaction on delivering the electric energy accepted at the closing of the market. Also, environmental benefit is expected with mitigation of the deviation from the assumed compromises by a better finding of offers able of fulfillment, less spinning reserved is needed, less thermal units are needed and less fossil fuel is used. Eventually, cloud-based solutions will allow approaches in order that when a power producer exploiting a wind-CSP systems at site has renewable energy available is all captured to be converted into electric energy injected without concerns on dynamic stability into the power grid in due time.

State of the Art

The literature on optimal scheduling is still rapidly growing. Coordination strategies and optimization approaches for short-term scheduling have been are in research, for instances: a envisaged wind-thermal coordination via simulated annealing unveiled an improvement of smoothing the active power fluctuations [START_REF] Chen | Simulated annealing-based optimal wind-thermal coordination scheduling[END_REF], and one via MILP unveiled a reduction of the uncertainty in wind power output [START_REF] Kamalinia | Generation expansion planning in wind-thermal power systems[END_REF]; a envisaged coordination of wind power with compressed air energy storage unveiled an improvement due to the synergies between wind power and compressed air energy storage [START_REF] Daneshi | Security-constrained unit commitment with wind generation and compressed air energy storage[END_REF]; a envisaged wind and hydro power coordination, specially pumped-storage hydro power systems [START_REF] Sanchez De La Nieta | Optimal coordinated wind-hydro bidding strategies in day-ahead markets[END_REF][START_REF] Khodayar | Enhancing the dispatchability of variable wind generation by coordination with pumped-storage hydro units in stochastic power systems[END_REF], unveiled an improvement on minimization of curtailment, energy imbalance and dispatchability. Furthermore, electrical vehicles connection to the power grid are expected to be used as controllable loads and a convenient operation with suitable market design may help in the accommodation more wind power [START_REF] Li | Emission-concerned wind-EV coordination on the transmission grid side with network constraints: Concept and case study[END_REF].

Coordination strategies have unveiled progress in reducing of the uncertainty of wind power output by the use of different dispatchable resources or loads, even though resources may not be installed in the same region of wind power systems deployment. But, when the surrounding of the load centres has attractive conditions of wind speed and solar irradiation, wind-CSP is an option design to be considered. This option enables a better accommodation of energy integration into the power grid and day-head electricity markets [START_REF] Usaola | Operation of concentrating solar power plants with storage in spot electricity markets[END_REF], as well as to increase the dispatchability attribute on the accommodation. The scheduling proposed in this paper is based on a MILP approach, which has been a successful approach for solving scheduling problems in general. So, also is expected to be one to help support decisions in a day-head electricity market.

Problem Formulation

The scheduling is computed by the maximization of the objective function given by the profit subject to transmission and systems constraints.

Objective Function

The profit is equal to the revenues from day-ahead electricity market sales and wind production incentive rate minus the CSP variable costs during the time horizon of the schedule. The objective function to be maximized is given as follows:

∑ ∑ ∑ ∈ ∈ ∈       - + - = K k C c CSP k c c I i Wind k i b k s k da k p p p p F , , ) ( β ξ π (1)
In ( 1

Constraints

The constraints for the scheduling are due to the transmission grid, operation of wind systems, operation or minimum up/down time of CSP systems.

Transmission grid

The transmission grid constraints are given as follows:

k p p p p C c CSP k c I i Wind k i b k s k ∀ + = - - - ∑ ∑ ∈ ∈ - , , 1 ) 1 ( ) 1 ( ψ ψ (2) k p p b k s k ∀ ≤ - - - ≤ - - χ ψ ψ χ ) 1 ( ) 1 ( 1 (3) k p p C c CSP k c I i Wind k i ∀ ≤ + ≤ ∑ ∑ ∈ ∈ χ , , 0 (4)
k y p k s k ∀ ≤ ≤ χ 0 (5) k y p k b k ∀ - ≤ ≤ ) 1 ( 0 χ (6)
In ( 2) to ( 6), ψ is the transmission loss, χ is the transmission capacity, and k y is a 0/1 variable. In (2), the electric power balance is enforced between the day-ahead electricity market trading with wind power systems and CSP systems. In ( 3) and ( 4), the electric power bounds are set for the transmission line. In ( 5) and ( 6), the energy flow in the line is set infeasible for simultaneously trading by selling and purchasing.

Operation of wind systems

The operational constraints of wind power system are given as follows:

k i W p k i Wind k i , 0 , , ∀ ≤ ≤ ( 7
)
k i P p Wind i Wind k i , 0 , ∀ ≤ ≤ (8)
where k i W , is the scheduled of wind turbine i in hour k, and

Wind i P
is the wind turbine i power bound. In [START_REF] García-González | Stochastic joint optimization of wind generation and pumped-storage units in an electricity market[END_REF], the operating limits for the scheduled wind power of each wind turbine i are set. In [START_REF] Chen | Simulated annealing-based optimal wind-thermal coordination scheduling[END_REF], the maximum power capacity of each wind turbine i is set.

Operation of CSP systems

The operational constraints of CSP systems are given as follows:

k c X p p p c SE k c FE k c CSP k c , , , , ∀ - + = (9) k c q p FE k c FE k c , , 1 , ∀ = η (10) k c q p SE k c SE k c , , 3 , ∀ =η (11)
k c u Q q q u Q k c PB c SE k c FE k c k c PB c , , , , , ∀ ≤ + ≤ (12) k c S q q k c FS k c FE k c , , , , ∀ ≤ + (13) k c q q q q SE k c FS k c S k c S k c , , , 2 1 ,
,

∀ - + = - η (14)
k c Q q Q S c S k c S c , , ∀ ≤ ≤ (15) k c Q q Q FE c FE k c FE c , , ∀ ≤ ≤ ( 16
)
k c P p CSP c CSP k c , 0 , ∀ ≤ ≤ (17) 1 ,..., 0 , 1 , , - = ∀ ≤ - + K k c RD p p T c SE k c SE k c (18) 1 ,..., 0 , ) ( , 1 , 2 - = ∀ ≤ - + K k c RU q q T c FS k c FS k c η (19) k c z M p k c SE k c , 0 , , ∀ ≤ ≤ (20) k c z M p k c FS k c , ) 1 ( 0 , , ∀ - ≤ ≤ (21) k c q q p p SE k c FE k c SE k c FE k c , 0 , , , , , , , ∀ ≥ (22)
In ( 9) to ( 21 z , is the 0/1 variable equal to 1 if TES c discharges power in hour k. In (9), the electric power balance is enforce between the power output of CSP system with the electric power produced from the SF, the storage and the parasitic power needed for maintaining the molten-salt fluid in operation conditions. This parasitic power occurs even if a CSP system is not operating, eventually implying that if the producer has not enough energy available, a small quantity of energy is soak up from the grid, incurring on an associated cost. The parasitic power is assumed constant. In [START_REF] Daneshi | Security-constrained unit commitment with wind generation and compressed air energy storage[END_REF] and [START_REF] Sanchez De La Nieta | Optimal coordinated wind-hydro bidding strategies in day-ahead markets[END_REF], the electric power from the SF and TES is considered dependent on the efficiencies associated with the thermal power of the SF and the storage power, respectively. In [START_REF] Khodayar | Enhancing the dispatchability of variable wind generation by coordination with pumped-storage hydro units in stochastic power systems[END_REF], the bounds for the sum of the power from the SF and TES are set. In ( 13) and ( 14), the balance of the thermal power in the SF and the energy stored in the TES is set, respectively. In [START_REF] Cplex | Solver manuals[END_REF], the bounds for the thermal power storage in the TES are set in order to avoid the solidification of salts and the maximum storage capacity to be exceeded. In [START_REF] Catalão | An artificial neural network approach for short-term wind power forecasting in Portugal[END_REF], the bounds for the thermal power from the SF are set. In [START_REF]Solar Advisor Model User Guide for Version 2.0[END_REF], the bounds for the power output of the CSP systems are set. In ( 18) and ( 19), the charge and the discharge ramp rates of the TES are set, respectively. In ( 20) and ( 21), power restrictions are set to prevent simultaneous discharging and charging of the TES in the same hour, imposed by the 0/1 variable, k z .

Minimum up/down time of CSP systems

The minimum up/down time constraints of CSP systems are given as follows:

K L k c u u u T SF k c k r UT k r r c r c T SF c , , 1 , ) ( , 1 , 1 1 , , K + = ∀ ≤ - + ∈ ≥ + - = - ∑ + (23) K F k c u u u T SF k c k r DT k r r c r c T SF c , , 1 , 1 ) ( , 1 , 1 , 1 , K + = ∀ - ≤ - + ∈ ≥ + - = - ∑ + (24)
In ( 23) and ( 24), the minimum up/down times for the CSP system c are set based on [START_REF]Market operator of the electricity market of the Iberian Peninsula[END_REF], where ] , [ min

T SF c T SF c UT K L + + = and ] , [ min T SF c T SF c DT K F + + =
.

Case Study

The case study is intended to show the advantages of the coordination between wind power systems and CSP systems having TES. These advantages are coming from the synergies between both systems and the adequate optimal mix of synergies is in favor of the proposed approach. The MILP for the approach has been solved using CPLEX 12.1 solver under GAMS environment [START_REF] Cplex | Solver manuals[END_REF]. A computer with 8 GB RAM with 2.30 GHz of CPU is used for the simulations of realistic case studies based on a Iberian wind-CSP system with 40 wind turbines,

and carried out with technical data shown in Table 1.

Table 1. Wind-CSP system data. All wind power systems have the same data as well as CSP systems, respectively. The installed wind power capacity is 80 MW and for CSP is 100 MWe. The systems share a transmission line connecting to the power grid. This line has 3 % of losses and exported power is assumed to be between 50 MW and 130 MW. The module efficiencies for the CSP systems are respectively: 1)

Wind i P / Wind i P (MW) CSP c P / CSP c P (MWe) S c Q / S c Q (MWht) FE c Q / FE c Q (MWt) 0 / 2 0 / 50 45 / 700 0 / 150 PB c Q / PB c Q (MWt) S c q 0 , (MWht)
40 . 0 1 = η ; 2) 35 . 0 2 = η ; 3) 80 . 0 3 = η
. The parasitic power is 3.5 MWe and the wind production incentive rate is assumed to be of 35 €/MWh.

The time horizon in the simulations is a day on an hourly basis, corresponding to a participation in a day-ahead electricity market. Within the time horizon are considered as input data not only the solar and the wind power profiles, but also the day-ahead market prices profile. The solar irradiation profile derived from the System Advisor Model [START_REF] Catalão | An artificial neural network approach for short-term wind power forecasting in Portugal[END_REF] was converted into available thermal power. The wind power profile is derived from the ANN forecaster designed in [START_REF]Solar Advisor Model User Guide for Version 2.0[END_REF]. The wind power and thermal power output profiles respectively are assumed to be the same for each wind turbine and CSP systems as is shown in The scheduling problem for the wind-CSP coordination has a day-ahead time horizon, due to this short horizon, the energy market prices can be assumed as deterministic input data given by forecast average prices. The prices of the Iberian electricity market given in [START_REF]Market operator of the electricity market of the Iberian Peninsula[END_REF] are used in the case study and shown in Fig. 2. A summary of the optimizing characteristics for processing the problem presented in the case study in what regards the number of constraints, continuous variables, binary variables and computation times is shown in Table 2. The number of installed wind turbines and CSP systems having TES are 20 and 2, respectively. The total installed capacities of wind power systems and CSP systems having TES are 80 MW and 100 MWe, respectively. The energy and profit for the transmission line capacity of 3. MW. Table 3 shows that an increase in the transmission line capacity of the line allows to make a better use of the energy stored, allowing an augmented profit.

The optimal schedule for the wind-CSP coordination in what regards the assessing of the impact of transmission line constraints with Fig. 3 (a) shows for the high price hours a significant CSP production in comparison with low price hours. The benefit of TES shifting the production is revealed and the excess energy eventually overloading transmission from hours 8, 10 to 12 and 16 is able to be stored for a convenient discharge at hours 11 to 13, 17, 18 and 24. Additionally, the synergies between wind energy and solar energy allows for the possibility of enhancing the scheduling due to the negative correlation. Thus, an efficient energy schedule is obtained, illustrating the proficiency of the optimization approach for accommodating this deployment with different changing patterns of renewable energy in a power grid. The thermal energy storage shows the significance of the line capacity regarding the storage in the TES. For instance, the increase in the line capacity raises the level of the storage in the low prices hours with solar thermal power in order to be conveniently used during the other hours. If the capacity factor is incremented by downsizing the transmission capacity, then curtailment is to be expected and consequent decrease of the energy produced. The thermal energy storages in function of the transmission capacity and of the time are shown in Fig. 4.

Conclusions

An approach based on MILP is proposed for the coordination problem of wind-CSP having TES systems and for supporting decisions of participating in day-ahead markets, taking into account transmission line constraints and prevailing technical operating ones for the wind power, CSP and TES. The approach reduces uncertainty and improves the integration of more energy into a grid due to the convenient optimal mix of synergies. The TES is effectively handled to improve the operational productivity. The case studies are in favour of the approach to support decisions in day-ahead markets.

  ), K is the set of hours in the time horizon, da k π is the forecasted day-ahead market price in hour k, s k p is the power sold to day-ahead electricity market in hour k, b k p is the power purchased from day-ahead electricity market in hour k, I is the set of wind turbines, ξ is the wind production incentive rate, Wind k i p , is the power output of the wind turbine i, C is the set of CSP systems, c β is the variable cost of the CSP system c and CSP k c p , is the power output of the CSP system c in hour k.

  the power produced by the solar field (SF) c and the TES c in hour k, c X is the parasitic power of the CSP plant c, 1 η is the SF efficiency, FE k c q , is the thermal power from the SF c in hour k, 3 η is the molten-salt tanks efficiency, SE k c q , is the storage power in TES c to produce electricity in hour k, thermal power bounds of the power block of CSP plant c, k c u , is the CSP plant c commitment in hour k, FS k c q , is the thermal power from the SF c stored in hour k, k c S , is the thermal power produced by the SF c in hour k, S k c q , is the thermal energy stored in TES c at the end of hour k, 2η is the TES efficiency, are the ramp-up and rampdown limits for charging and discharging the stored energy of TES c, M is a

Fig. 1 Fig 1 .

 11 Fig 1. SF thermal and wind power profiles.

Fig 2 .

 2 Fig 2. Day-ahead energy market prices.

  the enforced system are shown in Table

  shown in Fig.3(a) and (b).

  0

Fig 3 .

 3 Fig 3. Coordination for transmission capacities: (a) 60 = χ MW; (b) 130 = χ MW.

Fig 4 .

 4 Fig 4. Thermal energy storages in function of transmission capacity and over the time horizon.

Table 2 .

 2 Optimizing characteristics of the case study.

	#	Constraints	Continuous variables	Binary variables	CPU time (s)
	Case study	1,436	864	120	8

Table 3 .

 3 Energy and profit in function of transmission capacity.

	#	Transmission capacity (MW)	Energy stored (MWh)	Energy sold (MWh)	Profit (€)
	Case study	60 130	6,405 6,269	1,259 1,544	83,527 95,673

Table 3

 3

			allows a comparison between the profits, revealing a reduction of about 15% for
	χ	=	60

Acknowledgements. This work was supported by Fundação para a Ciência e a Tecnologia (FCT), through IDMEC under LAETA, Instituto Superior Técnico, Universidade de Lisboa, Portugal.