Irene Cheng

Anup Basu

Randy Goebel
email: goebel@cs.ualberta.ca

Interactive Multimedia for Adaptive Online Education †

Keywords: Interactive multimedia, subject knowledge and cognitive skills, student modeling, adaptive testing, educational games

Introduction

Interactive multimedia content has the capability to improve learning performance by enhancing user satisfaction and engagement. Multimedia content also provides better concept representation, which is not possible in conventional multiple-choice and fill-in-the-blank formats. In this article, we present a broader view of multimedia education especially for the years to come. Our vision is to provide publicly accessible education anywhere, at anytime and to anyone. We propose an innovative CROME (Computer Reinforced Online Multimedia Education) framework integrating the main components of education including learning, teaching and testing, as well as adaptive testing and student modeling. Our research is conducted in collaboration with industry and is in response to the industrial need and public support for "Online Multimedia Education." By presenting our research and development achieved in the last two years, we hope to share our experience with other researchers and bring awareness to the much-needed outcomes that have not received adequate attention. Our current implementation provides the groundwork upon which more robust, effective and far-reaching multimedia architecture can be developed.

The unique contributions of our system are: Automatic difficulty level estimation for selected subjects, compared to estimates based on user group calibration [1];

Adaptive testing using innovative item formats versus using traditional multiple choice items (Moodle's multiple choice, GMAT).

Computer-assisted automatic question item authoring, rather than manually creating different questions [START_REF] Chang | Next generation content representation, creation, and searching for new-media applications in education[END_REF];

Custom-designed modules for tracking precise concepts that are lacking in recommendation systems [START_REF] Adomavicius | Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions[END_REF]; and, Ability for testing beyond only subject knowledge, unlike other e-learning systems [4, 5, 6].

The Mechanism of Education

Historically, education was a commodity affordable only by the rich. This would still be the case for high school studies should free tuition not be offered by some governments. However, inequality of opportunities continues because the rich can afford private schools and tutors. The rich can access additional guidance outside classrooms while the poor have to rely on public school resources. To hire a qualified private tutor in North America, the regular hourly rate on curriculum specific subjects currently range from $40 to $60. In some municipalities, the difference in expenses between attending a private school and a public school can be thousands of dollars per month. The demand for affordable private tutors has created a recent trend in online tutoring, which allows students to reserve specific time slots in advance and, with selected tutors, get help with solving curriculum related questions. The students pay a set monthly fee for a certain number of tutoring hours. Since the service is provided online, the tutors can be remotely located. For example, some service providers take advantage of the lower operating cost overseas and hire tutors in countries like India to answer questions posted online by students in North America. In this kind of out-sourcing service, the issues to address include time difference, curriculum diversity, and the possible language and cultural barriers. Moreover, this approach requires little to no multimedia content because the focus is on the conversation between the tutor and the student either in a cyber chat room or through emails.

A more widely adopted and effective approach to promote global education is through the use of digital multimedia and e-learning. The format of e-learning can be as general as an open access web site like that supported by BBC [4] and Discovery Education [5], or curriculum specific programs provided by universities and institutions for high stakes exams, diplomas, undergrad and graduate exams. One example is the distance education programs provided by the University of Phoenix, La Salle and Athabasca University (AU) [6]. The European Union (EU) supported educational research program "new perspective for learning" [START_REF]New Perspectives for Learning: European Union Educational Research[END_REF] has helped develop and integrate many components for e-learning and develop learning societies under frameworks FP4 and FP5.

Instead of making educational material accessible online, tutorial DVDs containing multimedia content are also available in computer stores and other outlets. Multimedia education [START_REF] Allen | The Web: Interactive and Multimedia Education[END_REF] proves to be more effective than the traditional printed material because the dynamic and interactive settings can inspire students to learn and remain interested. Furthermore, abstract concepts are expressed better using acoustic or visual effects, animation and graphics, including 3D navigation. Human computer interaction provides instantaneous feedback to students and helps them feel engaged, like in a game-like environment. Since high speed computers and networks are commonplace in modern societies, online delivery has become an efficient tool to overcome geographic diversity. The use of a common language in cyberspace can narrow the gap in language and cultural barriers. For this reason, our research and development focus is on delivering Online Multimedia Education to promote education anywhere, at anytime, and to anyone.

Pervasive Multimedia Education

Online applications have been gaining broad acceptance among the general public, especially after the launch of semantic web and social web such as Second Life, YouTube and Facebook. Companies like Amazon, Google, Yahoo! offer very innovative and competitive online services largely because of user demands. The increasing acceptance of online applications and products by all ages is creating a unique opportunity to make a global impact with online education. There is an abundance of literature in the educational multimedia area covering a variety of topics from games to learner modeling. For example, computer games, played by an individual or collaboratively, have been widely used to teach concepts [START_REF] Feng | Joyce: A Multi-Player Game on One-on-one Digital Classroom Environment for Practicing Fractions[END_REF][START_REF] Galvao | Modeling Reality With Simulation Games for a cooperative learning[END_REF][START_REF] Mitchel | The Use of Computer and Videogames for Learning" Learning and Skills Development Agency[END_REF]; visual effects are used to assist learner understanding [START_REF] Lyvers | A Unique Instructional Tool for Visualizing Equipotentials and its Use in an Introductory Fields Course[END_REF][START_REF] Stubbs | Kana no Senshi (Kana Warrior):A New Interface for Learning Japanese Characters[END_REF]; artificial intelligence techniques have been used to recommend research papers to learners [START_REF] Tang | Utilizing artificial learners to help overcome the cold-start problem in a pedagogically-oriented paper recommendation system[END_REF]; active learner modeling [START_REF] Mccalla | Active learner modelling[END_REF] has been used to derive information on a peer group learning together, possibly in a collaborative environment; tele-mentoring is carried out through collaborative agents [START_REF] Friedman | Telementoring as a Collaborative Agent for Change[END_REF]; student performance is explored on a multimedia exam program [START_REF] Hao | Exploring Student Performance on a Multimedia Exam Program (Vexam)[END_REF]; and open exams have been set up for MBA/Business school admission [START_REF] Syvum | Computer-Adaptive Test for GMAT[END_REF]. However, most of the literature addresses learning applications of multimedia. The use of multimedia in testing has been relatively limited, compared to learning and training. For those systems supporting testing, most of the test questions still adhere to traditional styles, e.g. True-false, Multiple-choice and Fill-in-the-blank, and are not as novel as those proposed in our framework. In addition, our design integrates different aspects of education and is not restricted to a single target component.

CROME Framework: Examples of Pervasive Multimedia Learning and Testing Items

In order to illustrate the pervasive power of using multimedia content online and its impact on improving the quality of education, we will show some examples from our CROME implementation. We refer to a curriculum specific question as an "item" following the terminology adopted in Educational Psychology and Measurement. In the following, each example item can be used either in learning or testing depending on the log-on status: a practice or testing session respectively. Tutorials, hints and correct answers are available to the students in a practice session. In early development, "multimedia" referred mainly to digital images and videos. 3D objects and scenes became an important component only later. One application of 3D content is in chemistry, which can be categorized into three levels: macroscopic level, symbolic level, and atomic level [START_REF] Johnstone | Why is science difficult to learn: Things are seldom what they seem[END_REF]. A chemical phenomenon at the macroscopic level is normally observed in a laboratory and can be shown in videos for learning in addition to the scheduled lab session. The other two levels can be exposed in a more active and interesting manner, instead of writing a formula, e.g., O H NaCl HCl NaOH

3D Items

2 + → + , (1)
or drawing a molecule on paper. The understanding of molecular structures and the change of structures during a chemical reaction, such as breaking bonds inside a molecule, is best presented in a 3D context. Figure 1 shows screen shots of two Chemistry items. Students can use the mouse buttons to pick the required atoms from the periodic table, and drag them onto the canvas. They can then create the appropriate bonds between atoms, as well as navigate (translate, rotate, and zoom) through the atoms and molecules in 3D space. Atomic and molecular symbols in Formula (1) are replaced by 3D objects. A similar idea can be used to create Biology items as well. 3D objects have long been used in the entertainment industry and research, but their use in educational applications is still limited. We believe that it will not be long before tele-surgery and virtual reality tools become more accessible in education, allowing students to practice dissecting a virtual frog instead of using a real animal, and understanding the human body better by manipulating the different organs in virtual space. A commonly used interaction design is "drag and drop," which allows students to drag text or graphics to the appropriate locations on the screen. An example of a geography item requiring students to drag the correct labels onto the map is shown in Figure 2 (top, left). Drag and drop items can be used in other subjects as well. Figure 2 shows three more examples.

Drag and Drop Items (2D and 3D)

Logical-Mathematical Items

An interactive environment provides more entertainment, and thus motivation, to students when using and analyzing numbers. In the Math item shown in Figure 3 (top), a student needs to distribute the numbered objects into multiple baskets so that the sum in each basket is the same. Another Math item (Figure 3, bottom) requires the student to rotate the dividing lines on the pie chart so that the sizes of the colored portions correspond to the given ratios.

Language Items

Acoustic and visual effects can be combined to help students relate semantics to spellings and pronunciations. For example, consider different ways of presenting an item to learn and test vocabulary and spelling: Figure 4 shows one example. Note that a student can use the alpha-numeric virtual keyboard displayed on the screen, or use the computer keyboard. Three components are associated with this item: pronunciation (which can be obtained by pressing a button), spelling, and object recognition.

Educational Games

Online games have become commonplace not only among teens but also among adults. Our goal is to present education items in a way similar to what is attracting them to games, so that students can still enjoy their satisfaction through playing games, as well as benefit from learning. Education items can be used standalone or in a batch like activities grouped together in a game. A final score is awarded at the end of the game or when there is a time-out. The game also keeps the top scores to challenge participants. The balloon shooting game (Figure 5, top left) tests students' Math skills such as perfect squares, multiples, and prime numbers. A student has to shoot balloon, provided with sound effect, marked with the target number. The difficulty of the game is controlled by the speed of the balloons, the range of numbers displayed, and also the amount of distraction introduced into the background.

The budgeting game (Figure 5, top right) is a different way of testing Math skills. Here a student is given a budget to buy certain commodities. The student can click any item, which is individually priced, in the scrollable panel on the right and place it in the middle panel (shopping cart). The money spent on the selected commodities is displayed at the top of the screen. Crossword puzzles are used to test the vocabulary of a language as well as the semantics associated with a word. An example of such an item is shown in Figure 5, bottom. A student first relates a word with an image. By clicking on the image, the corresponding location of the word in the puzzle area is highlighted. The student then uses the virtual keyboard or the computer keyboard to spell out the word in the highlighted space. Web-based multimedia systems can be used in a broad variety of applications, e.g. hurricane analysis and simulation [START_REF] Cheng | A Reliable Web-based System for Hurricane Analysis and Simulation[END_REF]. CROME is designed for multimedia education and uses a combination of web-development tools in order to optimize the constrained resources and to provide user satisfaction. The development kits include Java 2D/3D applets, Javascript, Flash, J2ME, PHP and MySQL. SQL is an efficient query language used in many database retrieval systems [START_REF] Rishe | SemanticAccess: Semantic Interface for Querying Databases[END_REF]. The choice aims at platform and browser independence. While a Javascript item is fast to display, its 3D and animation capability is not comparable to Flash and Java3D. Flash animation is appealing but its 3D capability is not as good as Java3D. However, Java3D items need more programming skills for development, and can be slow during rendering when the content gets too complex. We choose one of these three for a particular item design depending on the complexity of the item and the intended outcome. The multimedia data flow in the CROME framework is shown in Figure 6. Among others, an important feature of our design is scalability. Similar to the multiple-choice format, a generic template is designed for each category of items, which share certain similarities. Multiple questions can be generated by altering the content inserted into the template. Furthermore, the layout, such as the number of objects and baskets shown in Figure 3, is controlled by parameters. This approach reduces recoding for individual questions.

An Overview of the CROME Multimedia Framework and its Innovative Components

Automatic difficulty level estimation

In the multiple choice or true/false format, an answer can only be correct or wrong. There is no partial mark awarded. However, when multimedia content, such as 3D items are used in Computer Adaptive Testing, a student's performance can be evaluated more accurately by considering partial scores. In order to evaluate the correctness of an answer and award partial marks, we interpret a molecular structure as a graph, where nodes are atoms and edges are bonds. In this way, we can assess the correctness of an answer by comparing the similarity between two graphs. A number of graph similarity matching algorithms can be found in the literature. Among these algorithms, graph edit distance [START_REF] Kelly | Graph edit distance from spectral seriation[END_REF] is commonly used. In this algorithm, a set of graph edit operations is defined. These edit operations include deletion, insertion, and substitution of a node or an edge. The edit distance of two graphs is defined as the length of the shortest sequence of edit operations required to transform one graph to the other. In our scoring scheme, we extend the edit distance to a weighted version. A scoring matrix is used to store the weights. More details on this approach can be found in [START_REF] Wu | An Interactive 3D Environment for computer based education[END_REF].

The graph based strategy for difficulty level estimation for chemistry questions discussed in [START_REF] Wu | An Interactive 3D Environment for computer based education[END_REF] may not be applicable for other subjects. A parameter-based strategy is a more general approach for assigning initial difficulties to items. We use Math questions as examples to illustrate the concept. For example, when solving the question "distribute the numbers so that the sum in each bin is equal" (Figure 3, top), the difficulty level of a question is defined by a function

) , ( nbr n bkt n f
, where n bkt is the number of baskets used and n nbr is the number of objects to distribute. The difficulty level increases as n bkt or n nbr increases. We verified the feasibility of our approach by conducting evaluation experiments. We have shown that it is possible to predict the difficulty level of the items in Figure 3, top, by using a 2-Parameter Logistic Model following Item Response Theory (IRT) [START_REF] Embretson | Item Response Theory for Psychologists[END_REF][START_REF] Linden | Handbook of Modern Item Response Theory[END_REF] and estimating the parameters based on time taken to solve a problem by a small group of students using linear regression.

Details on this approach can be found in [START_REF] Cheng | An algorithm for automatic difficulty level estimation of multimedia mathematical test items[END_REF]. Conventional use of multimedia content in education focuses on subject knowledge but not on cognitive skills. According to Gardner, each person has seven intelligence aspects [START_REF] Gardner | Frames of Mind: The Theory of Multiple Intelligences[END_REF]: linguistic, logical-mathematical, spatial, bodily-kinesthetic, musical, interpersonal, and intrapersonal. By employing multimedia items, we are able to test a student's broader range of skills. An example item to test one's musical intelligence is shown in Figure 7. The idea in this item is to evaluate how well a student can associate body movements with musical rhythms. The student has to match the videos labeled from <1> to <5> with the corresponding musical clips. Each clip can be played by clicking a music button on the left panel. Details on this work can be found in [START_REF] Cheng | Assessing Rhythm Recognition Skills in a Multimedia Environment[END_REF]. Complex and tedious processes are often encountered in curricula such as physics, chemistry, and mathematics, where the final answer is built upon the results of numerous smaller processes. We incorporate a Process Analyzer [START_REF] Cheng | Self-tutoring, Teaching and Testing: An Intelligent Process analyzer[END_REF] in our CROME framework. The objective is two-fold: to assist students improve their problem solving skills using step by step hints and instructions, and to assist teachers in monitoring student performance so that proper help can be provided in time. The Process Analyzer defines the top-level process as the root of a hierarchy composed of simpler and easier steps so that students who have difficulty solving the problem on their own are able to follow the hints and instructions leading to the correct answer. The hints and the scores in each step are controlled by educators. Every response given by a student is recorded for performance analysis and student modeling. The Process Analyzer comes with a graphical user interface (Figure 8), which provides an engaging and motivating learning environment.

Testing Beyond Subject Knowledge

Adaptive Testing and Student Modeling using Multimedia Items

Each composite or simple question in the multimedia item pool is designed for a given grade level and is assigned a difficulty value. A student is evaluated based on their success in obtaining a pre-defined percentage of correctness when answering questions of a particular difficulty. During the process, a student can be given easier or more difficult questions, depending on whether his or her current response is wrong or correct. This approach of adaptively selecting the next item has been used in multiple-choice, true/false and fill-in-the-blank type of questions. Using multimedia items in adaptive testing is still a subject of research: there is significant complexity in assigning difficulty values to multimedia items, and in creating a sufficiently large pool of effective items to evaluate student abilities.

Following an adaptive testing strategy described above and a step-by-step record of a learner, a training set can be accumulated over time. Machine learning and artificial intelligence techniques can be applied to analyze data in the training set. Since different students require different types of assistance, teachers can customize their guidance based on individual learning patterns. In addition to evaluating a student's ability level, an important contribution of our design is the monitoring of a student's progress. Moreover, the training set contains valuable information for curriculum designers to fine-tune education materials. Note that the sense in which individual students are "modeled" is directly related to the richness of the items being investigated.

Multimedia Item Authoring Graphical Interface

In order to assist educators, who either cannot afford too much time in addition to their regular teaching or do not possess the required technical skills, to create new multimedia question items, we implemented multimedia item authoring templates. These templates are plug-ins integrated into the basic CROME framework. A screen shot of the authoring interface is shown in Figure 9. The authoring module can be run online for individual item upload, or offline for subsequent upload in a batch. There are two main design challenges: First, the uniqueness of each innovative item type, and second, the need to provide different processing pipelines for online and offline item creation. Uniqueness of each item type means that each template has its own set of properties, and has its unique way of processing them. These challenges are overcome by separating the interface implementation into easy to manage logical components (Fig. 9). Although the authoring module is designed for educators, students can also use the tool to create questions, and through this process they will improve their understanding of the subject.

A demo of the CROME framework can be seen at [30].

Summary

In this article, we reviewed past and present trends in pervasive multimedia education. We showed with examples how the CROME framework was designed in order to address the vision of multimedia education anywhere, at anytime and for anyone.

Although there is research in the literature focusing on certain target areas in education, our framework is unique in the way it incorporates automatic difficulty level estimation, item generation, process analysis and testing beyond subject knowledge. We optimize the available multimedia resources and development kits to implement portable and scalable education items that are appealing and entertaining to the learners. The design of CROME aims to achieve portability, reusability, scalability and interoperability:

Portability -The implementation is based on Java which is platform independent, including J2ME on mobile devices.

Reusability -An object-oriented design makes individual components reusable.

Interoperability -An authoring tool is an integral part of the CROME system, sharing the item type design, as well as media and item database.

Scalability -New item type plug-ins can be added without the need to modify the basic framework or authoring module.

The example items presented in this article represent only a small portion of the pervasive multimedia ideas. Specific outcomes that cannot be tested purely using paper-and-pencil, such as inter-personal skills and remote collaboration, can now be tested in an online computer-based multimedia context. Using our framework as an initial step, we intend to bring awareness to the multimedia research community and to solicit effort in topics, which have not been adequately studied. In future work, we plan to develop collaborative learning environments over cell phones (Figure 10) that will allow students from anywhere around the globe to collaborate with anyone at anytime. Although cell phones have been extensively used for information exchange, making multimedia educational items available on cells in a collaborative learning and adaptive testing, given limited mobile resources including processing speed and bandwidth, is still a research challenge.

Figure 1 :

 1 Figure 1: (Top) An item showing a 3D molecule used to test atomic and molecular structure in chemistry; (Bottom) an item that requires a description of the molecular structural changes in a chemical reaction.

Figure 2 :

 2 Figure 2: Four example items using drag and drop: (top, left) the student has to drag the correct description to the right location on the map; (top, right) drag the right labels into the answer boxes, (bottom, left) drag organs to positions on the body, and (bottom, right) drag grocery products into correct categories.

Figure 3 :

 3 Figure 3: (Top) A Math item requiring a student to distribute numbers into four baskets so that the sum in each basket is the same. (Bottom): A Math item requiring a student to divide the pie chart into correct portions.

Figure 4 :

 4 Figure 4: An item requiring a student to spell a word and drag the corresponding image into the answer box.

Figure 5 :

 5 Figure 5: (Top, left): A Balloon Shooting game to practice and test a student's Math skills; (Top, right): A Budgeting game to test Math skills;(Bottom) A Crossword Puzzle to test the vocabulary of a language and the understanding of a word. The top scores are displayed before the game starts to challenge participants.

Figure 6 :

 6 Figure 6: The multimedia data flow in the CROME framework.

Figure 7 :

 7 Figure 7: (a) An example of a musical item to test a student's ability to perceive, express and transform musical forms, and (b) a sequence of video expressing different musical composition.

Figure 8 :

 8 Figure 8: A Chemistry example: A screen shot of our Process Analyzer.

Figure 9 :

 9 Figure 9: An example of the CROME authoring interface.

Figure 10 :

 10 Figure 10: Example item on a cell phone: (Top) A student needs to fill in the missing numbers so that the sum in each row and column is the same. (Bottom) Another item type "word scramble" showing the process of selection of alphabets to form the correct word.

Acknowledgements

The support of the CROME team is gratefully acknowledged.

The support of iCORE (Alberta Informatics Circle of Research Excellence), NSERC and Castle Rock Research are gratefully acknowledged.


Meghna Singh

Mrinal Mandal
email: mandal@ece.ualberta.ca

Anup Basu

Visual Gesture Recognition for Ground Air Traffic Control using the Radon Transform

Keywords: Gesture recognition, pose recognition, Radon transform, robot tele-operation

Human gesture recognition is an active topic of vision research which has applications in diverse fields such as collaborative virtual environments and robot tele-operation. We propose a novel method for the recognition of hand gestures, used by air marshals for steering aircraft on the runway, using the Radon transform. Various aspects of the algorithm, including acquisition, segmentation, labeling and recognition using the parametric Radon transform are addressed in this paper. A binary skeleton representation of the human subject is computed. The Radon transform is used to generate maxima corresponding to specific orientations of the skeletal representation. Feature vectors are extracted from the transform space by computing the normalized cumulative projections of the Radon transform on the angle axis. Kmeans clustering is then applied to recognize static gestures from the extracted features. This technique has the potential to provide information about the exact orientation of gesture segments and can find use in ground control of unmanned air vehicles. Experiments with image data corresponding to the various ground air traffic control gestures used in directing aircrafts, highlight the potential application of this approach.

I. INTRODUCTION

The increasing interaction between man and machine has lead to research in the area of man-machine interfacing. The various approaches to this interfacing can be divided into intrusive (such as haptic devices and data gloves) and non-intrusive approaches (based on computer vision techniques). These two approaches have their own pros and cons. The intrusive approach provides a higher spatial and temporal resolution to the man-machine interface, while the non-intrusive approach is more practical for cases where it is difficult to interact with devices such as markers and haptic gloves. Human gesture recognition is one of the many domains of man-machine interfacing and involves understanding and recognition of the gestures involved in either interpersonal communication (sign language) or man-machine communication (specific dictionary of gestures/poses for robot tele-operation).

Robot tele-operation has been conducted in the past via sensor gloves, keyboard interface and joysticks. Other approaches have been to learn a particular dictionary of visual commands, which are acquired and acted upon by the robot. In the latter approaches, gesture recognition is an integral part of non-verbal communication with the robot. Iba et al. [START_REF] Iba | An Architecture for Gesture Based Control of Mobile Robots[END_REF] use a data glove and HMMs to recognize six hand gestures that are used to control a robot. Fong et al. [START_REF] Fong | Novel Interfaces for Remote Driving: Gesture, Haptic and PDA[END_REF] use gesture driven control and haptic devices for robot teleoperation. Their gesture commands however appear to be view dependent such that forward and reverse commands will be distinguishable only when viewed at an offset angle. In non intrusive works, Ugalde et al. [START_REF] Garcia-Ugalde | Segmentation and Classification of Hand Gestures for Man-machine Communication[END_REF] classify segmented hand gestures based on the correlation coefficient of wavelet coefficients of trained and test sequences. Hu et al. [START_REF] Hu | Visual Gesture Recognition for Human Machine Interface for Robot Teleoperation[END_REF] use a set of seven gestures to issue commands such as turn left, start and stop. They evaluate various existing techniques to determine the most suitable algorithm for recognition. Bradiski et al. [START_REF] Bradski | Motion Segmentation and Pose Recognition with Motion History Gradients[END_REF] use motion history images to compute the current gesture of the object for recognition. Chang et al. [START_REF] Chang | Vision Based Tracking and Interpretation of Human Leg Movement for Virtual Reality Applications[END_REF] use Mealy machines and Markov processes to interpret commands issued by human leg movement for virtual reality applications. The above-mentioned techniques have either high complexity thus making them unsuitable for real time operations or have low correct recognition rates. Some of the approaches such as haptic devices and data gloves are intrusive, and are not suitable for a more dynamic environment. In this paper, we propose a visual gesture recognition algorithm based on the Radon transform of skeletal lines of the arm gestures/poses. The proposed technique provides a high correct recognition rate and has low complexity. We also present a particular application of our proposed method in ground air traffic control. Another anticipated application is gesture based control of robots and unmanned aircrafts/vehicles. This paper is organized as follows. In Section II we review the Radon transform. We present our proposed technique for pose recognition in Section III. Performance evaluation is presented in Section IV. Finally, conclusions are presented in Section V.

( ) ( ) ( )dxdy y x r y x f r f α α δ α sin cos , , - - = ℜ ∫ ∫ ∞ ∞ - ∞ ∞ - (1)
where r is the perpendicular distance of a line (r=xcosα +ysinα ) to the origin and α is the angle formed by the distance vector to the line (see Fig. 1(a)). The delta function in [START_REF] Iba | An Architecture for Gesture Based Control of Mobile Robots[END_REF] evaluates to a non-zero value only for points that lie on this line. The RT is essentially the description of a function in terms of its integral projections (see Fig. 1 (b)). For example, two parallel lines in the image space will translate to two maxima at the same angle α but different distances from the origin r (Fig. 1 (c) and (d)) in the Radon transform space.

In the past, Radon transform (RT) and Hough transform (HT) have been used in various pattern recognition applications such as line detection, facial expression recognition and symbol recognition [START_REF] Leavers | Use of Two-Dimensional Radon Transform to Generate a Taxonomy of Shape for the Characterization of Abrasive Powder Particles[END_REF][START_REF] Tsapatsoulis | On the use of Radon Transform for Facial Expression Recognition[END_REF][START_REF] Tabbone | Technical Symbol Recognition Using the Two-dimensional Radon Transform[END_REF]. It has also been proven that while the HT can be considered a special case of the RT in continuous space, they are non-identical in discrete form [START_REF] Hansen | Fast Curve Estimation Using Preconditioned Generalized Radon Transform[END_REF]. The deficiencies with using the HT are the distortion in shape and location of peaks in transform space, and problems in transforming straight lines that pass through the origin. Also, the standard implementation of the discrete HT via accumulator bins creates issues. If the bins are too small then intersection of the transform curves of collinear image points may fall in separate bins, and if the bins are too large, then parallel lines close together will lie in the same bin. It is because of these associated issues with the HT that we use the RT instead, in our technique for gesture recognition. To the best of our knowledge the RT has never been used before for human gesture recognition.

III. PROPOSED TECHNIQUE

In this section we describe the proposed technique for gesture recognition using the RT. The flowchart in Fig. 2 illustrates the primary steps of this technique (these are described in detail in the following subsections). Our methodology depends on the following two assumptions: 1. For the purpose of gesture recognition, any gesture of the arms can be replaced by the medial skeleton representation of that gesture. 2. The representation of the medial skeleton of a gesture in RT space provides information concerning the classification of that gesture. We acquire images in which subjects perform the arm gestures corresponding to ground air traffic control gestures. These images are processed to compute a binary representation of the arm segments in each image. Medial skeletonization is then applied to these binary images to obtain a skeleton representation of each gesture. The parametric representation of these skeleton images is obtained by using the RT. Feature vectors are subsequently extracted from the Radon transform and clustered for the recognition of unknown gestures. The proposed technique can be applied to a large number of applications such as gait analysis and robot-teleoperation. However, in this paper we demonstrate the application of the proposed technique for ground air traffic control.

Gesture recognition decision

Feature Extraction and Clustering

Image acquisition and preprocessing

Medial skeletonization

Radon transform computation

Gesture recognition decision

Feature Extraction and Clustering

Image acquisition and preprocessing

Medial skeletonization

Radon transform computation

A. Image Acquisition and Preprocessing

The proposed algorithm is designed to recognize eight ground air traffic control gestures. Images are acquired such that they correspond to the gestures indicated in Fig. 3. We impose restrictions such that the motion of the upper torso and legs is minimal and most of the motion occurs only in the arms. More robust techniques such as human body models can also be implemented to acquire motion of just the arm segments; however, complexity will increase with the addition of a body model. Once the images have been acquired, the foremost step of any vision based recognition algorithm is background-foreground separation. We perform background separation by using a statistical background modeling approach. We assume a relatively stationary background (assuming noise at each pixel to be time varying) and use an adaptive threshold This preprocessing generates binary images for arm poses, which are morphologically processed to fill any holes that may be present in the foreground (Fig. 4).

B. Medial Skeletonization

Morphological operation called thinning is then applied to these binary images to obtain the medial skeleton of each segment (Fig. 4). We can now consider these medial skeleton images of size Y X × to be a special discrete function ( ) (

y x f S ,
)    = otherwise skeleton of part if y x f S , 0 , 1 , (2)

C. Radon Transform Computation

Subsequent to the extraction of the skeletal lines, the RT is used to detect the orientation of these lines. The discrete version of the RT of skeletal images is then computed as the following summations: denotes the Kronecker delta function. It is observed in RT space (Fig. 5) that there are maxima corresponding to the location and angle of the medial skeleton in image space. In real cases the maxima are not very localized and tend to be slightly diffused. This is because the skeletons are rarely straight lines but rather a collection of connected line segments varying slightly in their orientation (Fig. 5). Feature vectors must now be extracted from the RT space for further processing.

] { } [ ] { } ( ) α α τ , min , max 2 1 r r ℜ - ℜ = (4)
The Radon transform space is then thresholded as follows:

If [ ] τ α ≥ ℜ , r then [ ] [ ] α α , , r r ℜ = ℜ else [ ] 0 , = ℜ α r .
Since it is desired that this technique be translation independent we compute the cumulative projections of the Radon transform on the α axis denoted hence by cumRT as follows:

[ ] [ ] ∑ = ℜ = r i i cumRT 0 ,α α (5)
We also normalize these projections ( normRT ) such that the distributions lie between 0 and 1 (see Fig. 6). [START_REF] Chang | Vision Based Tracking and Interpretation of Human Leg Movement for Virtual Reality Applications[END_REF] We implement K-means clustering with K=8 clusters, one cluster for each gesture that is to be recognized. From the set of acquired image data, 8 images (one for each gesture) are selected as the known gesture set. The remaining image data is selected as the unknown or test data set. All steps of the algorithm (Fig. 2) till K-means clustering are computed for both the known and unknown gesture sets. The clustering however is done only on the normalized cumRT of the unknown gesture set. In order to recognize the gestures that are clustered together, we compute the nearest neighbor of the cluster centroid in the known gesture set.

IV. PERFORMANCE EVALUATION

In order to evaluate the performance of our proposed method, we captured test images of size 640x480 using a SONY DCR-PC100 CCD camera. The images were then converted into 8-bit gray scale format using MATLAB. Five subjects were asked to perform randomly a set of arm gestures from the ground air traffic control gesture set. Subsequent to image acquisition the proposed algorithm was implemented in MATLAB6.5 R13 on a Pentium IV 3.2GHz processor, 1GB RAM.

Overall 86 test images were used for performance evaluation and 76 of those test images were correctly recognized giving a correct recognition rate (CRR) of 88% (see Table-I). For relative performance comparison we also implemented two techniques proposed by Hu et al. [START_REF] Hu | Visual Gesture Recognition for Human Machine Interface for Robot Teleoperation[END_REF] for gesture recognition, which include template matching (via block matching, geological transform) and skeletonization (via vectorization and angle computation). The recognition rates achieved with [START_REF] Hu | Visual Gesture Recognition for Human Machine Interface for Robot Teleoperation[END_REF] are also indicated in Table I. Fig. 8 shows two of the test images that were incorrectly classified. It is observed that due to poor segmentation and thinning the medial skeleton is no longer a unitary segment but has loop like regions. These act as noise sources and cause a larger than usual Radon transformation. Thus when the Radon transform is thresholded (see section III D) these noise sources are not eliminated and get incorporated into the cumulative Radon transform and subsequently the normalized cumRT. Thus, noisy feature vectors lead to misclassification. However, these loop formations were rare and did not bias the clustering to one particular class. As part of future work, we plan to address this problem. It was also observed during evaluation that the actual RT representation of gesture for 'stop' was much different from ideal (see Fig. 9), but to our advantage it was also much different from all the other gestures used. Therefore, gestures corresponding to stop were clustered satisfactorily.

V. CONCLUSIONS

In this paper, we have proposed an algorithm for the recognition of human arm gestures using the Radon transform. A particular application of recognizing ground air traffic control gestures is identified and tested. While there are other template-based approaches in contemporary literature that are being used for gesture recognition, these approaches are computationally more expensive and also require normalization of the images. Our approach on the other hand does not require any normalization of the image itself. The promising recognition rates achieved with the parameterized Radon transform emphasize the feasibility of the proposed method. In the future, we plan to use the orientation information derived from the Radon transform space to reconstruct motion for virtual models.

Fig. 1 :

 1 Fig.1: (a) Pictorial representation of line used in (1), (b) Illustration of RT as an integration of the image; RT of two parallel lines -(c) Skeleton image, (d) Mesh plot of RT of parallel lines in (c). II. RADON TRANSFORM J. Radon first introduced the Radon transform (RT) in a classic paper in 1917. The RT of a function ( ) y x f , , can be defined as

Fig. 2 :

 2 Fig.2: Flowchart of proposed technique.

3 :

 3 Ground air traffic control gestures as designated by the United States Air Force [11]. (a) Move ahead, (b) Slow down, (c) Reverse, (d) Turn Left, (e) Stop, (f) Turn Right, (g) Reverse Left, and (h) Reverse Right.

  Each pixel of the current frame is thesholded against the corresponding pixel of the background model to extract foreground information. The mean intensity ( ) i is the pixel value at location ( y x, ) in the 'i th ' frame. The threshold for each pixel in the background model is calculated using the following equation:To obtain the foreground we classify each pixel in frame 'k' ( N k > ) according to the following inequality:

Fig. 4 :

 4 Fig.4: Sample images showing (a) original data, (b) binary image data and (c) results of the medial skeletonization operation.

Fig. 5 :Fig. 6 :

 56 Fig.5: RT of skeletal lines. (a) Actual data and its diffused RT, (b) Ideal data and its sharp RT.

[

Fig. 7 :Fig. 8 :

 78 Fig.7: Scatter plot representation of the performance of the proposed algorithm for the first 43 test images. The horizontal axis represents the unknown gesture image number. The vertical axis represents the eight gestures that were identified. The script next to each test image point represents the actual gesture performed in the image. The legend for the gestures is as follows: RT: Right Turn, ST: Stop, RR: Reverse Right, LT: Left Turn, MA: Move Ahead, RL: Reverse Left, RV: Reverse, SW: Slow. The images whose gestures were erroneously clustered are indicated with a cross symbol ⊗, all other recognition decisions are correct. TABLE II: CONFUSION MATRIX OF THE ACTUAL AND PREDICTED GESTURES. PREDICT→ ACTUAL↓ RT ST RR LT MA RL RV SW RT 9 1 0 0 0 0 0 0 ST 0 6 0 0 2 2 0 0 RR 0 0 10 0 0 0 0 0 LT 0 0 0 8 0 2 0 0 MA 0 0 0 0 10 0 2 0 RL 0 0 0 0 0 10 0 0 RV 0 0 0 0 1 0 11 0 SW 0 0 0 0 0 0 0 12

TABLE 1 :

 1 RECOGNITION RATE OF PROPOSED ALGORITHM. CRR: CORRECT RECOGNITION RATE.

	True	# of	Correct recognition rate in %
	gesture	test
		images	Template	Vectors [4]	Proposed
			matching [4]		RT
	Move ahead	12	100	75	80
	Right turn	10	100	100	90
	Reverse right	10	40	100	100
	Reverse left	10	40	80	100
	Reverse	12	0	100	92
	Left turn	10	100	100	80
	Stop	10	100	0	60
	Slow	12	50	100	100
	Overall	86	65%	82%	88%
	Fig. 7 shows a scatter plot representation of the performance
	of the proposed algorithm for 43 test images. The horizontal
	axis represents the test image number of the unknown gesture
	image. The vertical axis represents the eight gestures being
	identified. The legend for the gestures is as follows:
	RT: Right Turn, ST: Stop, RR: Reverse Right, LT: Left Turn
	MA: Move Ahead, RL: Reverse Left, RV: Reverse, SW: Slow
	It can be seen from Fig.7 that most of the gestures in the
	unknown images were clustered to their correct known
	gesture. The images whose gestures were erroneously
	clustered are indicated with a cross symbol ⊗. A confusion
	matrix of the results is presented in Table II.

ACKNOWLEDGEMENT

The participation of volunteers in the experiments is gratefully acknowledged.


Mehdi Faraji
email: faraji@ualberta.ca

Irene Cheng
email: locheng@ualberta.ca

Iris Naudin
email: iris_naudin@yahoo.fr

Anup Basu
email: basu@ualberta.ca

Segmentation of Arterial Walls in Intravascular Ultrasound Cross-Sectional Images Using Extremal Region Selection

Keywords: Intravascular, Ultrasound, IVUS, Segmentation, Extremal Regions, Extremum Level, EREL

Intravascular Ultrasound (IVUS) is an intra-operative imaging modality that facilitates observing and appraising the vessel wall structure of the human coronary arteries. Segmentation of arterial wall boundaries from the IVUS images is not only crucial for quantitative analysis of the vessel walls and plaque characteristics, but is also necessary for generating 3D reconstructed models of the artery.

The aim of this study is twofold. Firstly, we investigate the feasibility of using a recently proposed region detector, namely Extremal Region of Extremum Level (EREL) to delineate the luminal and media-adventitia borders in IVUS frames acquired by 20 MHz probes. Secondly, we propose a region selection strategy to label two ERELs as lumen and media based on the stability of their textural information. We extensively evaluated our selection strategy on the test set of a standard publicly available dataset containing 326 IVUS B-mode images. We showed that in the best case, the average Hausdorff Distances (HD) between the extracted ERELs and the actual lumen and media were 0.22 mm and 0.45 mm, respectively. The results of our experiments revealed that our selection strategy was able to segment the lumen with ≤ 0.3 mm HD to the gold standard even though the images contained major artifacts such as bifurcations, shadows, and side branches. Moreover, when there was no artifact, our proposed method

Introduction

Catheter-based Intravascular Ultrasound (IVUS) has captured considerable attention in the last two decades. This worldwide attention is mostly due to the ability of the imaging method to picture the inside of the human coronary arteries and, hence, provide an opportunity to diagnose and treat cardiovascular diseases such as atherosclerosis (e.g., thin-cap fibroatheroma) that causes a heart attack and a brain stroke [START_REF] Frostegård | Sle, atherosclerosis and cardiovascular disease[END_REF]. Aside from this, the IVUS technique can be helpful in visualizing some internal structures of the human coronary such as the lumen, and thickness and distribution of the plaques [START_REF] Katouzian | A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images[END_REF]. Therefore, IVUS is regularly used to locate the atherosclerosis lesions in the coronary arteries to study the lumen and plaque dimensions, and to guide intervention and stent deployment [START_REF] Ma | A review of intravascular ultrasound-based multimodal intravascular imaging: the synergistic approach to characterizing vulnerable plaques[END_REF].

A typical IVUS imaging system consists of four parts: catheter, transducer, pullback device, and scanning console. The catheter is composed of a 150 cm long guidewire and a tip of 1.2-1.5 mm in size. It is usually inserted in the femoral artery and proceeds toward the coronary arteries. The catheter is responsible for carrying the ultrasound transducer, or other necessary devices, such as inflatable balloons and stents [START_REF] Katouzian | A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images[END_REF]. The transducer is a miniaturized ultrasound probe that emits ultrasound pulses and listens for the backscattered signal. After the catheter has reached the distal end of the coronary, it needs to be manually or automatically pulled back. The speed of the pullback varies between 0.5-1 mm/s [START_REF] Katouzian | A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images[END_REF]. The scanning console is essentially a computer used to post-process the acquired signals (using amplification, filtering, etc.) to provide a user-friendly environment for the surgeon to control the device.

Segmentation of the acquired IVUS images is among the most challenging tasks in medical image analysis. In particular, delineating the interior (lumen) and exterior (media) vessel walls is problematic due to the presence of various artifacts such as motion of the catheter after a heart contraction, guide wire effects, bifurcation and side-branches or similar echogenicity between the vessel wall and some plaques. In some cases even the difference in transducer frequencies affect the segmentation results [START_REF] Katouzian | A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images[END_REF].

The intrinsic difficulty of IVUS segmentation has attracted many researchers to study and develop solutions using different methodologies, such as intensitybased, statistics and probability-based, active contour and graph search-based approaches. In addition, several methods have been proposed to segment either the lumen or the media or both. A great number of approaches in the literature have utilized the 2D information provided as cross-sectional frames to segment the lumen and media. These 2D cross-sectional gray-scale images are formed after digitization of the backscattered RF signals and are called IVUS B-mode frames. To the best of our knowledge, recent approaches have mostly worked on the B-mode frames which will be reviewed in the following paragraph. For more in-depth reviews of the methods published before 2013, please refer to [START_REF] Katouzian | A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images[END_REF][START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF].

The lumen and media-adventitia border variations have been modeled within a shape space in [START_REF] Unal | Shapedriven segmentation of the arterial wall in intravascular ultrasound images[END_REF]. The lumen segmentation is then performed by maximizing a nonparametric probability density energy. Also, the edge information has been used to segment the media-adventitia. A physics-based model of the IVUS signal scattered by the structure of the vessel has been used in [START_REF] Mendizabal-Ruiz | A physics-based intravascular ultrasound image reconstruction method for lumen segmentation[END_REF] to estimate the differential backscattering cross-sections from the IVUS RF signal. The segmentation curve is obtained after training a Support Vector Machine (SVM) model using the annotated data. Deformable models have been used in [START_REF] Taki | Automatic segmentation of calcified plaques and vessel borders in ivus images[END_REF][START_REF] Zhu | A snake-based method for segmentation of intravascular ultrasound images and its in vivo validation[END_REF][START_REF] Mendizabal-Ruiz | Segmentation of the luminal border in intravascular ultrasound b-mode images using a probabilistic approach[END_REF] to detect the border of lumen/intima and media/adventitia. In [START_REF] Taki | Automatic segmentation of calcified plaques and vessel borders in ivus images[END_REF] anisotropic diffusion followed by an edge detector are used to create an initial segmentation which is then corrected using both geometric and parametric deformable models. In [START_REF] Zhu | A snake-based method for segmentation of intravascular ultrasound images and its in vivo validation[END_REF] an improved version of gradient vector flow (iGVF) has been proposed which includes a balloon force in the snake model that lets the contour pass over leaks and bifurcations. A probabilistic method that formulates the deformation of a lumen contour curve and can be minimized has been proposed in [START_REF] Mendizabal-Ruiz | Segmentation of the luminal border in intravascular ultrasound b-mode images using a probabilistic approach[END_REF]. However, every first frame of the sequence needs user interactions to manually segment the lumen and media. Following this, a SVM is trained over the annotated data to compute the probability that each pixel is blood or non-blood. A fourfold algorithm based on a deterministic statistical strategy for segmenting the media has been proposed in [START_REF] Zakeri | Automatic media-adventitia ivus image segmentation based on sparse representation framework and dynamic directional active contour model[END_REF]. Their method consists of preprocessing, initial contour detection, active contour segmentation, and contour refinement.

First, a sparse binary image is constructed using the local appearance model and the initial contour is elicited. To achieve this, a feature vector is built for each pixel. It includes gray-level values of the pixel's neighbours, the average intensity of neighbours and the gray-level values of the pixel's neighbours in a contour-enhanced version of the image [START_REF] Zakeri | Automatic media-adventitia ivus image segmentation based on sparse representation framework and dynamic directional active contour model[END_REF]. The K-SVD method is utilized to classify the extracted feature vector. The initial contour is then refined. Next, an active contour model is used to delineate the media border in polar coordinates. The detected contours are then refined using the information provided by identifying the calcification and shadow regions. Artificial Neural Networks have been employed in [START_REF] Su | An artificial neural network method for lumen and media-adventitia border detection in IVUS[END_REF] to represent the spatial and neighbouring features of the IVUS image data. As a result, two different vascular structures for lumen and media are extracted and optimized using two ANNs. The borders obtained are then refined and smoothed by an active contour model. In [START_REF] Yan | A novel segmentation approach for intravascular ultrasound images[END_REF] the lumen is segmented by a combination of image gradient and fuzzy connectedness model and the media-adventitia border is extracted by a fast marching model. A sequential forward selection process using SVMs and PR curve has been employed to conduct an in-depth analysis of several image features in [START_REF] Vercio | Assessment of image features for vessel wall segmentation in intravascular ultrasound images[END_REF]. It has been shown that the median filtered image and Haralick's texture features [START_REF] Haralick | Statistical and structural approaches to texture[END_REF] provide stronger discrimination capabilities for arterial structures. A limitation of their analysis is that it only works for artifact-free IVUS sequences.

As we can see, most approaches have employed either a type of energy minimization method or require annotated data in order to train a classifier or an ANN. However, in this paper we propose a straightforward approach that not only does not require training but also does not use any variational method or deformable model. We show that by extracting EREL features [START_REF] Faraji | Extremal regions of extremum levels[END_REF][START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF] the problem of the IVUS segmentation can be relaxed to a region selection. In particular, we illustrate that it is very likely to find regions similar to lumen and media among the extracted ERELs. Therefore, we propose a selection procedure that efficiently chooses ERELs that are most similar to lumen and media.

The rest of this paper is organized as follows. In Section 2 we present our proposed method and describe the sequence of Intravascular Ultrasound images that we use throughout this paper. Section 3 illustrates the segmentation results of our proposed method. In Section 4 we discuss the advantages and weaknesses of the proposed method. Finally, the concluding remarks are given in Section 5.

Materials and Method

Materials

Our proposed method has been evaluated on the test set of publicly available dataset consisting of 326 in-vivo pullbacks of the human coronary artery frames that were acquired by the Si5 (Volcano Corporation), equipped with a 20 MHz Eagle Eye monorail catheter [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF]. The dataset includes a multi-frame 3D context that has between 20 to 50 gated frames acquired using a full pullback at the enddiastolic cardiac phase from 10 patients. Manual annotations for IVUS images are available in the dataset. The annotations have been provided by four clinical experts who work regularly with IVUS echograph. The experts were not aware of other expert's manual annotations and two of them repeated the annotations after about one week from their first delineation [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF].

The test set contains several types of common artifacts. Specifically, it includes 44 images containing bifurcation, 93 images with a side vessel artifact, and 96 images that have been contaminated by a shadow artifact (some images contain more than one artifact). There are also 143 images that do not contain any serious artifacts except for plaque.

Proposed Method

In this paper we present a segmentation approach for 20 MHz Intravascular Ultrasound images based on a region detection strategy. Particularly, we investigate whether a recently proposed novel feature extraction method called Extremal Regions of Extremum Levels (EREL) [START_REF] Faraji | Extremal regions of extremum levels[END_REF][START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF] can segment the most essential regions of interest (lumen and media) from the IVUS images required to establish the atherosclerotic plaque area [START_REF] Destrempes | Segmentation method of intravascular ultrasound images of human coronary arteries[END_REF]. The proposed method consists of four steps. We first remove the typical artifacts of IVUS frames, such as ring-down effects and calibration squares. Then ERELs are extracted and the obtained regions are filtered based on their types. Next, we perform a region selection procedure to specify two regions as lumen and media. Finally, the contour of the two selected regions is traced and smoothed by an ellipse fitting algorithm.

Preprocessing

Most of the IVUS images have been contaminated by speckle noise [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF].

Speckle is a multiplicative noise that imposes difficulties in processing the Ultrasound images [START_REF] Loizou | Despeckle filtering algorithms and software for ultrasound imaging[END_REF]. Therefore, to decrease the sensibility of our method to speckle noise, we first use a non-linear median operator to filter the IVUS images.

One of the main identifiable artifacts in IVUS images is the ring-down effects of the catheter that need to be eliminated from the B-mode frame or from its polar image. Otherwise, there is a high risk of obtaining an erroneous segmentation. To remove the ring-down effects of the catheter we employ the method proposed in [START_REF] Unal | Shapedriven segmentation of the arterial wall in intravascular ultrasound images[END_REF] which is a very fast and straightforward procedure. Detecting the ring-down artifact can be done by processing the whole volume since the artifact is almost available in all of the IVUS frames. Therefore, taking the minimum over all the frames generates an image where there is a significant contrast between the artifact and the non-artifact pixels.

(a) (b) (c) (d) (e) (f) (g) (h)
I min (x, y) = min i∈λ I i (x, y) (1)
where λ is a set of available frames in a particular IVUS sequence. We can then locate the artifacts' coordinates by subtracting the artifact zones from every frame, as in [START_REF] Unal | Shapedriven segmentation of the arterial wall in intravascular ultrasound images[END_REF]. Another type of artifact that we can detect using Eq.1 is the calibration square artifact. These small squares have a very bright constant intensity in all frames that remains bright in the minimum image. In several longitudinal cuts of the IVUS volume the effects of these artifacts are revealed as horizontal lines.

Figure 1 illustrates both the artifacts and the resulting image after removing 150 them.

Extremal Regions of Extremum Levels

EREL [START_REF] Faraji | Extremal regions of extremum levels[END_REF][START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF] is a region detector that employs a union-find structure [START_REF] Sedgewick | Algorithms[END_REF] in conjunction with the edge information to detect a series of connected pixels from the image. The edge information of the image is included in the method by using the Maxima of Gradient Magnitude (MGM) points. The idea underlying EREL is to binarize the image with all possible integer thresholds and analyze the results obtained based on their global criterion and their local edge information.

The regions belonging to the globally distinguished levels (Extremum Levels) are then extracted from the union-find tree.

Generally, two types of regions can be extracted from a gray-level image. The first type includes regions that evolve from brighter surfaces to darker boundaries which are identified by Q -. The superscript ' -' emphasizes the fact that the intensity values are decreased from the surface of the regions towards the boundaries. The second type consists of regions that evolve from darker surfaces to brighter boundaries and are denoted by Q + . This type of the region is consistent with the inherent characteristics of the lumen and media visualized by the backscattered 20MHz IVUS signals. Therefore, we only need to extract Q + regions to obtain the lumen and media because both regions evolve from darker surfaces to brighter boundaries.

To use EREL, we need to set several initialization parameters, namely A min , A max , α, β. These parameters define the functionality of the detector and can be tuned based on the application [START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF][START_REF] Faraji | Extremal regions of extremum levels[END_REF]. In particular, we use A min , A max to set the minimum and maximum area of the extracted regions. To better separate small regions from bigger ones, we choose a value for A min that correlates with image dimensions. Specifically, we set A min = (R × C)/100 and

A max = (R × C)/3
where R and C represent the number of rows and columns of the IVUS image, respectively. The parameter α is usually in [0 2.5] and represents the strength of the resulting interest points [START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF]. In this study we set α = 0.5. Finally, β = 1 denotes the width of the moving window over the global criterion vector [START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF]. The extracted EREL regions using the above-mentioned values are illustrated in Figure 2. However, not all four types of the regions depicted in Figure 2 encompass lumen and media regions. In fact, we only need to extract large area Q + regions as illustrated in Figure 2(d). Since large area Q + regions contain the actual lumen and media segments, detectors need not track Q -regions and, therefore, omit unnecessary computations which eventually helps to have a faster detector.

EREL Selection

The goal of this section is to address the problem of finding the most appropriate ERELs to be designated as lumen and media. By the most appropriate we mean the closest regions to the gold standard. As can be seen in Figure 2(d),

although ERELs are nested regions, it is clear that there is at least one EREL that is very close to the true lumen and similarly there is at least one EREL that corresponds to the true media. Therefore, we can relax the problem of lumen and media segmentation to only a selection procedure, i.e., assigning two nested ERELs to lumen and media.

Local maxima searching is the approach that we employ to select lumen and media from the nested set of ERELs. Assume that we have a vector (denoted by V ) representing the evolution of the Q + regions. The index of each element of the vector V corresponds to a Q + region. Since the extracted ERELs start from the smallest enclosing region and end with the largest enclosing region, the actual region corresponding to the lumen should be found among the regions located at the early indices of the vector. Likewise, a region representing the media is expected to be found among the regions belonging to the end section of the vector V .

Our aim is to construct a vector that ultimately gives us the stability of the regions in terms of the length of the boundary, average intensity and entropy variation. As we can see in Figure 2(d), the boundaries of the Q + regions are not smooth and are subject to large variations. Therefore, lumen and media regions should be selected among those Q + regions that have more stable boundary length variations. Also, the average intensity of the Q + regions should be stable enough (i.e., they should not change much over several subsequent regions).

Entropy measure can be used in order to create a feature vector that is sensitive to textural information of the regions.

The calculation of these three features are straightforward. The boundary lengths of the regions are available as an output of the EREL algorithm and are calculated based on a bottom-up tracking of boundary pixels along a parametric curve C(p).

L = 1 0 | ∂C(p) ∂p |dp (2)
Additionally, the calculation of the average intensity of ERELs can be readily done.

E = N i=1 Q + i N (3)
where N represents the area of the region and Q + i is the intensity value of pixel i. The entropy measure of a grayscale region [START_REF] Haralick | Statistical and structural approaches to texture[END_REF] is denoted as follows.

H = - K i p i log 2 p i (4)
where p i is the value of the bin i of the normalized histogram of the region and captures the probability of having a pixel with a certain gray-value. K is the number of available bins in the normalized histogram.

Afterwards, for every IVUS image, we create a vector (V ) where each element is obtained from the product of the above-mentioned measures for each region.

V = L 1 E 1 H 1 L 2 E 2 H 2 ... L n E n H n ( 5
)
where n is the number of Q + regions extracted by the EREL algorithm.

Vector V illustrates the variation of textural information of regions through the extraction and evolution of Q + regions. It is strictly increasing because Q + regions are nested and non-repetitive. More stable sequences of vector V are more likely to represent lumen and media since these stable sequences shows that the extremal regions are subject to saturation and the subsequent regions might contain a noticeable change. The best way to find the stable regions is to create a vector describing the stability score of regions. Every element of the stability vector is calculated as follows.

Ω i = V i V i+1 -V i-1 (6)
where i specifies a specific element of the vector V and varies from one to the number of the detected Q + regions for an IVUS image.

The local maxima of the stability score points to regions with high stability because the ratio of their current value to the change among their two neighbours is larger than the other surrounding elements. So, we select lumen and media from the detected local maxima. Specifically, a Q + region with higher prominence value among the first two peaks is considered as lumen. If the IVUS image contains no artifacts, the media will be represented by the last detected peak. Based on our observation, the stability score of the images that contain serious artifacts have none or a small number of peaks since the presence of the artifacts interferes with the natural extraction of Q + regions (see Figure 3(ii)).

Therefore, when a small number of local maxima is detected, we consider the last extracted region as media. This process for an IVUS with no particular artifact is illustrated in Figure 3. The local maxima of the stability score indicates the regions for which the variation of the textural characteristics is more stable than their surrounding regions. As can be seen in Figure 3(i)(a), the second peak is selected as a suitable region for the lumen since it has a higher prominence than 235 the first peak. Also, the region corresponding to the last peak of Ω which is chosen as the media has been shown in Figure 3(i)(b).

It is important to note that before searching for the local maxima we need to remove outliers. To find outliers we employ the modified Z-score normalization suggested in [START_REF] Iglewicz | How to detect and handle outliers[END_REF].

M i = 0.6745(A i -Ã) M AD (7)
where A i represents the area of the region i, Ã is the median of a vector of all region areas, and M AD denotes the Median Absolute Deviation and is calculated by the following equation.

M AD = median(|A i -Ã|) (8)
Therefore, the regions that have a modified Z-score less than Z min = -3 and greater than Z max = 3 are unlikely to represent lumen and media and hence can be removed from the selection process.

Contour Extraction

Since the general shape of the lumen and media regions of the vessel are very similar to conic sections, we propose to represent lumen and media by ellipses.

To find the pixels inside and on the ellipse border, it is sufficient to find the orientation of the ellipse, the major and minor axis length. Generally, EREL outputs all pixels belonging to each extracted region in addition to its shape description parameters which are three coefficients of ellipse equation [START_REF] Faraji | Extremal regions of extremum levels[END_REF][START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF].

Specifically, if a region Q is described by an ellipse with three coefficients, namely c,d and e then:

∀x, y ∈ Q : cx 2 + 2dxy + ey 2 ≤ 1 ( 9
) (i) No Artifact
(ii) Plaque and Shadow There is a direct relationship between the parameters of the fitted ellipse and the second central moments [START_REF] Shapiro | Computer vision[END_REF]:

M =   c d d e   = 1 4(µ yy µ xy -µ 2 xx )   µ yy µ xy µ xy µ xx   ( 10
)
where µ xx , µ xy and µ yy are the second order central moments and are calculated as follows [START_REF] Shapiro | Computer vision[END_REF]:

µ xx = 1 A (x,y)∈Q (x -x) 2 (11)
µ yy = 1 A (x,y)∈Q (y -ȳ) 2 (12)
µ xy = 1 A (x,y)∈Q (x -x)(y -ȳ) ( 13
)
where A represents the area of the region and (x, ȳ) specifies the coordinates of the region's centroid.

After having obtained matrix M (reported by EREL), finding pixels belonging to ellipse border of the regions is straightforward. We assume that M = VΛV -1 where V denotes a matrix containing the eigenvectors and Λ indicates a matrix of eigenvalues. The minimum (λ min ) and maximum (λ max )

eigenvalues in Λ are used to calculate the length of the minor(b) and major (a)

axes of the ellipse, respectively [START_REF] Szeliski | Computer vision: algorithms and applications[END_REF].

a = 1 √ λ max (14) b = 1 √ λ min (15)
The orientation of the ellipse (θ) can be recovered by calculating the angle between the major axis and the x axis.

Let V = [u v] where u = [u 1 u 2 ] T and v = [v 1 v 2 ]
T are two column vectors corresponding to the minimum and maximum eigenvalues, respectively. The orientation of the ellipse is then obtained as follows.

θ = tan -1 v 1 v 2 (16)

Results

In this section, we present the segmentation results of our method. Also, by showing the best case results, we demonstrate that regions very close to the lumen and media exist among the extracted ERELs and a proper selection strategy (the proposed method) can distinctively select the lumen and media regions from the extracted ERELs. Furthermore, we present extensive evaluation results of our method based on three standard evaluation metrics on IVUS frames containing various artifacts.

Evaluation Measures

To assess the segmentation obtained by our method, we employ three evaluation metrics, namely Jaccard Measure (JM), Hausdorff Distance (HD), and Percentage of Area Difference (PAD). Using these metrics that have been also

used in [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF] to evaluate the results of 8 state-of-the-art methods on the same dataset makes it possible to draw a fair comparison between our method and the results reported in [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF].

The Jaccard Measure is calculated based on the comparison of the automatic segmentation result and the manual segmentation delineated by experts. It quantifies the overlap area between the automatic and manual segmentation as computed by the following equation.

JM = R auto ∩ R man R auto ∪ R man (17)
Where R auto is the vessel region segmented by the method and R man represents the region that has been segmented manually by experts.

The Hausdorff Distance between the automatic (C auto ) and manual (C man ) curves is the greatest distance of all points belonging to C auto to the closest point in C man and is defined as follows [START_REF] Molinari | Caudlesef: carotid automated ultrasound double line extraction system using edge flow[END_REF].

HD = max{d(C man , C auto ), d(C auto , C man )} (18)
To calculate d(C auto , C man ) first the minimum of all Euclidean distances from each point belonging to C auto to all points in C man is obtained. Then, d(C auto , C man ) is computed by taking the maximum of all the minimum distances. Similarly, d(C man , C auto ) is computed by taking the maximum of all minimum distances from C man to C auto [START_REF] Molinari | Caudlesef: carotid automated ultrasound double line extraction system using edge flow[END_REF].

The Percentage of Area Difference calculates the segmentation area difference between the automatic (A auto ) and manual (A man ) segmentation and is computed as follows.

P AD = |A auto -A man | A man (19)

Best Case Results

280

In order to show that the extracted EREL regions have the potential to represent lumen and media regions, we evaluate all of the extracted ERELs by calculating the evaluation metrics (Section 3.1) for the contours of each EREL and the manually annotated contours. Then, the ERELs that correspond to the maximum JM of lumen and media are selected as the best extracted ERELs for 285 that frame. The quantitative results in comparison with the intra-observer and inter-observer variability are reported in Table 2.

Discussion

The present study is twofold. We first investigate whether the extracted ERELs [START_REF] Faraji | Extremal regions of extremum levels[END_REF][START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF] are proper regions to be considered as lumen and media delineations of 20 MHz Intravascular Ultrasound frames taken from the inside of the human coronary arteries. We quantitatively (Table 2) validate that very close candidates can be found from the small number (almost less than 50) of extracted ERELs for each IVUS frame. Specifically, the best case results demonstrate the average of the closest extracted EREL to the lumen and media among all 435 images of the dataset [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF]. As the results of the best case study suggest, if we design a proper selection strategy we are able to find two regions out of the extracted ERELs that are very close to the actual lumen and media structure.

Secondly, we propose an ERELs selection strategy based on creating a vector that represents several textural characteristics of the regions and search for the regions which have higher textural stability scores. This method works very well when no artifact is present in the IVUS image or more accurately, when no major artifact is attached to the media. In this case, the average Hausdorff distance for both lumen and media is less than 0.3 mm which shows that the proposed method works very well for images without any strong artifact. Furthermore, the general performance of our proposed method for lumen segmentation is superior to other available methods in terms of the average Hausdorff distance, which is 0.3 mm. Likewise, our method is able to segment the lumen when the image contains shadows (HD = 0.29 mm) or side vessel (HD = 0.24 mm) artifacts, which is by far the lowest distance to the gold standard.

If artifacts that are usually dark (low intensity value) regions, join the lumen and media regions, a type of leak will be created. This leak significantly increases the size and the mean intensity of the regions and generates irregular patterns that are very difficult to distinguish by the EREL extraction algorithm. Some of the inaccurate segmentations that are caused by the presence of the artifacts are shown in Figure 5. As can be seen in Figure 5, although the presence of the artifacts disrupts the detection of the media regions, most of the lumen regions can still be segmented accurately. This is also supported by the quantitative results presented in Table 3. Evidently, even in the presence of artifacts such as bifurcations, side vessels, and shadows, the lumen segmentation performance remains high, though the accuracy of the media segmentation drops. In fact, even in the worst case condition of lumen segmentation that happens when the image contains bifurcation artifacts, the average Hausdorff distance to the gold standard is only 0.53 mm which is still lower than most of the other methods.

On the other hand, the media segmentation is more sensitive to the presence of artifacts. For instance, the Hausdorff metric for media in some cases increases to 1.22 mm for bifurcation and to 1.24 mm for shadow artifact. Accordingly, it can be concluded that the lumen segmentation is robust to the common artifacts of the IVUS images.

The success of the proposed method depends strictly on the correct extraction of ERELs. In cases when continuous and clear boundaries do not exist between extremal regions, EREL cannot extract correct candidate regions for lumen and media. For example, in IVUS images taken with a 40 MHz catheter probe, some lumen pixels located on the boundaries adjacent to intima do not have clear connections to each other. A similar situation occurs for the boundaries between media and adventitia. This creates leakages that attach media region to the adventitia. So, the sequential process of EREL's enumeration connects pixels from two semantically different regions and constructs a big connected component that includes both regions. Eventually, the leakage obstructs EREL's region extraction capability for detecting Q + regions. This leakage can also be created in IVUS 20 MHz images by various artifacts that somehow attach the media regions to the adventitia as illustrated in Figure 5.

Apart from the above, our proposed method offers several benefits. Not only is our method automatic, but works also solely on the B-mode images (in contrast to some methods available in the literature that work on the polar space [START_REF] Unal | Shapedriven segmentation of the arterial wall in intravascular ultrasound images[END_REF][START_REF] Taki | Automatic segmentation of calcified plaques and vessel borders in ivus images[END_REF][START_REF] Mendizabal-Ruiz | Segmentation of the luminal border in intravascular ultrasound b-mode images using a probabilistic approach[END_REF][START_REF] Zakeri | Automatic media-adventitia ivus image segmentation based on sparse representation framework and dynamic directional active contour model[END_REF]) and, thus, it eliminates the need for transformation to the polar space. In addition, some methods [START_REF] Sun | A parallel method for segmenting intravascular ultrasound image sequence[END_REF][START_REF] Cardinal | Intravascular ultrasound image segmentation: a threedimensional fast-marching method based on gray level distributions[END_REF] require the whole volume to be presented in order to segment the arterial walls. This makes it impossible to segment the lumen and media in online segmentation applications (during the in-vivo pullback procedures). On the other hand, our method delineates lumen and media contours using only the information available in the current frame and, hence, performs no redundant processing. Furthermore, due to the low computational complexity of the EREL [START_REF] Faraji | Extremal regions of extremum levels[END_REF][START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF], every frame can be segmented in linear time based on the number of pixels in the frame. The average run time per frame in the proposed method over the test set of [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF] is 0.19 seconds.

Conclusion

In this study, we investigated the suitability of a recently proposed region detector called Extremal Regions of Extremum Levels for detecting the lumen and media regions. The results of our experiments reveal that among the ERELs extracted from a single IVUS frame there exist regions with very low difference from the manually annotated data. Therefore, we presented a region selection approach to automatically segment the arterial walls from Intravascular Ultrasound images taken with a 20 MHz catheter probe. We embedded the mean intensity value, the entropy, and the boundary length of the regions into a single feature vector representing the texture information of the region. Then, our selection method looks for regions with higher stability score. The region that is most stable at the beginning section of the texture vector is considered as the lumen. Similarly, the last region with a stable textural variation is labeled as media.

We evaluated our segmentation method on the test set of a standard publicly available IVUS dataset using three common evaluation metrics; namely, JM, HD, and PAD. Extensive quantitative comparisons show the high accuracy of our proposed method. In general, the average HD for our lumen segmentation results is 0.3 mm which is the closest distance to the gold standard among all existing methods. In addition, when no artifact was present, our method segments both lumen and media achieving average HD lower than 0.29 mm and 0.31 mm, respectively.

As reported in the best case results, among the extracted EREL regions, there exist regions very close to the actual structure of the arterial walls. In future work, we would like to devise a better selection strategy that chooses ERELs more accurately.

Since IVUS is completely radiation free, unlike X-ray and CT, in the future we would also like to investigate the feasibility of combining the segmentation results from individual slices to obtain 3D views of collateral arteries.

Acknowledgment

We gratefully acknowledge the assistance of Prof. Simone Balocco at the University of Barcelona for providing the labeling information on the existing artifacts in the dataset.

• An accurate method for segmenting the vessel walls in IVUS 20 MHz images is proposed.

• Extremal Regions of Extremum Levels extracts similar regions to the arterial walls • Textural information of the ERELs is used to choose the proper regions • When there are no artifacts, both lumen and media are segmented very accurately • Our lumen segmentation is robust when various artifacts are presented in the image

Figure 1 :

 1 Figure 1: Artifact removal in B-mode and polar frames. a) A 40 MHz IVUS B-mode frame. b) Computed minimum image of (a). The yellow colour demonstrates higher values and the red colour represents lower values. c) Result of the IVUS frame shown in (a) after the artifact removal. d) A longitudinal cut of the whole volume. The horizontal lines are the effects of the artifact revealed after cutting. e) Corresponding polar frame of (a). f) Calculated minimum image of (e). The yellow colour demonstrates higher values and the red colour represents lower values. g) Corresponding polar frame of (e) after artifact removal. h) Result of artifact removal in all frames of the volume illustrated in a longitudinal view that is cut by the same plane as the one used to cut (d).

Figure 1 (

 1 b) and Figure 1(f) show the resulted minimum image in B-mode and polar frame respectively.145

Figure 2 :

 2 Figure 2: Extracted ERELs from a 20 MHz IVUS frame belonging to dataset [4]. The initial parameters of EREL are: α = 0.5, β = 1, A min = (R × C)/100 = 1474 and Amax = (R × C)/3 = 49152. a) Q -regions with small area. b) Q + regions with small area. c) Q - regions with large area. d) Q + regions with large area. Contour colours have been randomly assigned and are only for visualization purposes.

  240

Figure 3 :

 3 Figure 3: The evolution of the Q + regions and their stability criteria in presence of no artifacts vs. plaque and shadow artifact. a) The best candidate region representing the lumen. b) The best region representing the media. The neglected regions are highlighted by the yellow colour and the selected regions for lumen and media are indicated by magenta and green colour.

Figure 4 :

 4 Figure 4: Lumen and media segmentation results. Segmented lumen and media have been highlighted by magenta and green colours respectively. The yellow dashed lines illustrate the gold standard that have been delineated by four clinical experts [4].

Figure 5 :

 5 Figure 5: Several inaccurate lumen and media segmentation results in the presence of various artifacts. Segmented lumen and media have been highlighted by magenta and green colours respectively. The yellow dashed lines illustrate the gold standard that have been delineated by four clinical experts [4].

Table 1 :

 1 Comparison of each method's run time (required for segmenting a frame) reported in[START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF] and the proposed method.

		Category	Semi/Auto	2D/3D	Time per frame	Hardware used
	Proposed Method	Lumen and media	AUTO	2D	0.19 s	Core i7-4700HQ, 2.4 GHz
	Unal et al. [5]	Lumen and media	SEMI	2D	3.25 s	Pentium 6200 2.13 GHz
	Wang et al.[4]	Lumen	SEMI	2D	1 m 40 s	Xeon 2.67 GHz
	Destrempes et al.[4]	Lumen and media	SEMI	2D	8.64 a	Core i7 Q740 @ 1.73 GHz
	Downe et al. [23]	Lumen and media	AUTO	3D	0.16 s	Core 2, 2.4 GHz
	Alberti et al.[4]	Lumen	AUTO	3D	13 s	Core 2, Duo 2.13 GHz
	Ciompi et al. [24]	Media	AUTO	2D	20 s	Core i7, 2.8 GHz
	Mendizabal et al. [9]	Lumen	SEMI	2D	4.96 s	Core i7, 2 GHz
	Exarchos et al. [4]	Lumen and media	AUTO	2D	0.5 s	Core 2, Duo 3.33 GHz

Table 2 :

 2 The best case performance results of the proposed method. Measures represent the mean and standard deviation (std) evaluated on 435 frames of dataset[START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF]. The measures are categorized based on the presence of a specific artifact in each frame. The evaluation measures are Jaccard Measure (JM), Hausdorff Distance (HD), and Percentage of Area Difference (PAD).

				Lumen			Media
			HD	JM	PAD	HD	JM	Pad
		EREL	0.22 (0.12) 0.91 (0.04) 0.03 (0.03) 0.50 (0.45) 0.83 (0.15) 0.13 (0.15)
	General Performance	Intra-obs	0.28 (0.13) 0.88 (0.05) 0.11 (0.08) 0.24 (0.12) 0.92 (0.03) 0.06 (0.04)
		Inter-obs	0.17 (0.13) 0.93 (0.05) 0.04 (0.06) 0.14 (0.12) 0.95 (0.03) 0.03 (0.03)
		EREL	0.20 (0.09) 0.92 (0.03) 0.02 (0.02) 0.23 (0.17) 0.92 (0.05) 0.03 (0.05)
	No Artifact	Intra-obs	0.28 (0.13) 0.88 (0.05) 0.11 (0.08) 0.24 (0.12) 0.92 (0.03) 0.06 (0.04)
		Inter-obs	0.17 (0.13) 0.93 (0.05) 0.04 (0.06) 0.14 (0.12) 0.95 (0.03) 0.03 (0.03)
		EREL	0.36 (0.22) 0.85 (0.07) 0.07 (0.07) 0.95 (0.49) 0.67 (0.16) 0.27 (0.17)
	Bifurcation	Intra-obs	0.30 (0.12) 0.88 (0.04) 0.09 (0.06) 0.24 (0.09) 0.92 (0.02) 0.06 (0.03)
		Inter-obs	0.18 (0.21) 0.92 (0.07) 0.05 (0.09) 0.15 (0.13) 0.95 (0.04) 0.03 (0.03)
		EREL	0.20 (0.09) 0.90 (0.04) 0.03 (0.03) 0.62 (0.53) 0.78 (0.16) 0.18 (0.16)
	Side Vessels	Intra-obs	0.30 (0.13) 0.88 (0.05) 0.10 (0.08) 0.24 (0.11) 0.92 (0.04) 0.06 (0.04)
		Inter-obs	0.20 (0.11) 0.91 (0.05) 0.06 (0.05) 0.15 (0.10) 0.95 (0.03) 0.03 (0.04)
		EREL	0.22 (0.11) 0.89 (0.04) 0.03 (0.03) 1.10 (0.38) 0.64 (0.13) 0.31 (0.14)
	Shadow	Intra-obs	0.31 (0.13) 0.88 (0.05) 0.11 (0.08) 0.27 (0.15) 0.92 (0.04) 0.06 (0.05)
		Inter-obs	0.18 (0.14) 0.93 (0.05) 0.04 (0.06) 0.14 (0.10) 0.96 (0.03) 0.02 (0.02)

Computational Cost

The run time of the proposed method depends only on the complexity of EREL which is O(N ) [START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF], where N is the total number of pixels in the image.

The subsequent operation proposed in this paper in Section 2.5 works on a constant number of regions. The maximum number of nested regions that can be extracted from the same root pixel is at most 256 for an 8-bit image [START_REF] Faraji | Extremal regions detection guided by maxima of gradient magnitude, Image Processing[END_REF].

In our experiments with IVUS images, the number of candidate ERELs (Q + ) rooted from the center of image are even less (lower than 75). Therefore, the region properties denoted in Eq.2, Eq.3, Eq.4 and Eq.6 and the second moment matrix are calculated on a constant number of regions containing far less than N pixels. Therefore, the overall run time of the proposed method is O(N ) in the worst case. A comparison between the actual run time of the proposed method and the methods reported in [START_REF] Balocco | Standardized evaluation methodology and reference database for evaluating ivus image segmentation[END_REF] is shown in Table 1.

EREL Selection Results

Qualitative evaluations are illustrated in Figure 4 and show the successful segmentation results of the proposed EREL selection strategy for 20 IVUS frames.

290

The lumen areas are highlighted by the magenta colour while the media regions are green. Also, the manually annotated contours for both lumen and media are drawn as yellow dashed lines. As we can see, the chosen frames contain a variety of lumen and media morphologies.

A detailed evaluation result and comparison with 9 recently published IVUS 295 segmentation methods are reported in Table 3 where the performance of the proposed EREL selection strategy in the presence of various artifacts is shown as well.