Carlos Fernando Crispim-Junior

Véronique Joumier

Yu-Liang Hsu

Ming-Chyi Pai

Pau-Choo Chung

Arnaud Dechamps

Philippe Robert

François Bremond
email: francois.bremond@inria.fr

Veronique Joumier

Alzheimer's patient activity assessment using different sensors

Keywords: Information Technology, Health & self-esteem, multi-modal sensors monitoring systems, Alzheimer's disease, dementia, elderly, older people

à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche

INTRODUCTION

The older people population is expected to grow dramatically over the next 20 years, and the number of people requiring care will grow accordingly (including Alzheimer's patients), while the number of people able to provide care will decrease [START_REF] Zouba | An Activity Monitoring System for Real Elderly at HomeÇ Validation Study[END_REF] .

Information and Communications Technologies (ICT) have been proposed to improve and support older people care (e.g., wearable sensors, smart-homes, video monitoring systems). For instance, wearable sensors measurements have been proposed for medical diagnosis trials on the evaluation of older people motor functions [START_REF] Gillain | The value of instrumental gait analysis in elderly healthy, MCI or Alzheimer's disease subjects and a comparison with other clinical tests used in single and dual-task conditions[END_REF][START_REF] Auvinet | Dual tasking and gait in people with Mild Cognitive Impairment according to amnesic and non-amnesic subgroups, preliminary results[END_REF] . The patients wore a chest or wrist sensor during a gait analysis test to extract kinematic parameters for gait patterns analysis (e.g., stride length, stride cadence). The extracted kine-matic parameters were used as evidence to evaluate existing differences between gait patterns of health participants and patient diagnosed with dementia (Alzheimer's disease patients at mild to moderate stage). Similar ICT applications were applied and studied for the analysis of Parkinson's disease impairments, e.g., force-plates placed below patient gait test path [START_REF] Manap | Anomaly Gait Classification of Parkinson Disease based on ANN[END_REF] and accelerometer-based wearable devices [START_REF] Cancela | Gait assessment in Parkinson's disease patients through a network of wearable accelerometers in unsupervised environments[END_REF] . This approach could be also explored for the study of patient activity patterns in activities of daily living (e.g., dressing, eating). Although wearable sensors are suitable for description of personal kinematics parameters, they do not provide data about a subject's actions over his/her environment (contextual data). For example, a person' interaction with household appliances (coffee making machine, TV, etc), or his/her time spent in certain home areas (e.g., kitchen, bedroom, living room). Video-based system for older people surveillance is growing as a research field (particularly frailty detection) [START_REF] Foroughi | Intelligent video surveillance for monitoring fall detection of elderly in home environments[END_REF][START_REF] Nasution | Video surveillance for elderly monitoring and safety[END_REF][START_REF] Nasution | Intelligent Video Surveillance for Monitoring Elderly in Home Environments[END_REF][START_REF] Wang | An Intelligent Surveillance System Based on an Omnidirectional Vision Sensor[END_REF][START_REF] Fleck | Smart Camera Based Monitoring System and Its Application to Assisted Living[END_REF] , as it can provide data about a people interaction with their environment (e.g., time spent in zones and interaction with objects of interest). Applications are generally associated with detection of daily living activities (e.g., eating, dressing, walking), or the detection of (potentially) dangerous situations (e.g., older people falls). But, for particular domotic environment, illumination conditions, or camera field of view, these systems could have problems in characterizing a person's postures (e.g., bending to an object), or at quantifying an impairment in a limb movement (e.g., a restricted range of motion). Higher resolution video cameras or multiple cameras arrays could be applied in this case, but they will increase the ICT project cost and complexity.

To address the above limitations of current systems, we present the first results of the SWEET-HOME project, an initiative focused on the development of a medical and ICT-based system for the improvement of diagnosis and evaluation of dementia progress in early stage Alzheimer's Disease (AD) patients.

We attempt to overcome the described limitations of the sole use of video or wearable sensors by deploying several sensors types (video and accelerometer) for our clinical tests analysis. These sensors were tested inside a clinical protocol developed and executed by the Memory Center of the Nice Hospital -France and the Department of Neurology at National Cheng Kung University Hospital -Ta iwan.

The clinical protocol is intended to assess older people performance in IADLs (Instrumental Activities of Daily Living, e.g., preparing coffee, watching TV) and in gait analysis tests (e.g., performing a balance test). IADLs are preferred to ADLs, as IADLs have been recently addressed as better indicators of emerging neuropsychiatric symptoms [START_REF] Monaci | Neuropsychological screening performance and the association with activities of daily living and instrumental activities of daily living in dementia: baseline and 19-to 24month follow-up[END_REF] . ADLs generally refer to basic activities of personal self-care (e.g., eating, dressing, washing), while IADLs are associated with more complex tasks, e.g., using the telephone, shopping groceries, organizing medications, and managing personal finances. IADLs seem to be affected earlier than ADLs for early stage Alzheimer patients. We postulate that comparing data from several sensors could provide new (or at least complementary) quantitative evidence about changes in a patient's activities profile [START_REF] Monaci | Neuropsychological screening performance and the association with activities of daily living and instrumental activities of daily living in dementia: baseline and 19-to 24month follow-up[END_REF] . The clinical protocol' activities were specifically chosen according to their possibility of being realistically achieved into the observation room of the hospital and at the same time provide objective information about dementia symptoms.

Figure1. Video monitoring system results being analyzed by the evaluation platform (VISEVAL) developed by the STARS team (France).

MATERIALS AND METHODS

This multi-centric study was carried out in French and Taiwanese experimental sites. Experiments in France took place in an observation room equipped with household appliances. Experimental data was recorded using a 2D video camera (AXIS®, Model P1346, 8 fps -frames per second), and an ambient audio microphone (Tonsion, Model TM6, Software Audacity, WAV file format, 16bit PCM/16kHz). A motion sensor (e.g., MotionPod®) was fixed on the chest of the participant to quantify their movements. MotionPod® sensor provides an index of activity and estimation about the patient posture (standing, sitting, lying, walking), both of them with a resolution of 1 data per second.

The Taiwanese experiments took place in indoor and outdoor environments. For the indoor experiments a room equipped with household appliances was used and experimental data was recorded using eight ambient 2D video cameras (AXIS, Model 215PTZ, 30 fps). For outdoor experiments a tri-axial accelerometer mounted on the shoes of the participants was used to analyze their gait parameters

Clinical protocol

The Clinical Protocol is divided into four scenarios: directed activities (indoor), semi-directed activities (indoor), undirected activities (indoor), and directed activities (outdoor).

Scenario 01 (S1) or Directed activities in an indoor environment is intended to assess kinematic parameters about the participant's gait profile (e.g., static and dynamic balance test, walking test). During this scenario an assessor stays with the participant inside the room and asks him/her to perform seven physical activities within 10 minutes. These activities are briefly described as follows: Scenario 03 (S3) or undirected ("free") activities (30 minutes) aim at assessing how the participant spontaneously initiates activities in the room (e.g., reading magazines/newspapers, drinking, playing cards, and watching TV) and also how he/she organize time without receiving specific instructions. Figure 2 shows the French indoor observation room (Scenarios 1-3).


Scenario 04 (S4) or Directed activities in an outdoor environment aim at analyzing different gait parameters using the tri-axial accelerometers and the stride algorithm developed in the project. The participant is asked to walk around the ring region in the NCKU campus (following the plan shown in Figure3). During this walking period he/she performs a simple walking test of 40 m on a straight line; and a dual task test where he/she needs to walk the same distance while counting down from 100 to 1.

Systems overview

Table 5 describes the sensors used to compute different parameters about older people activities (e.g., duration of activity execution, gait parameters).

Automatic Sensor Monitoring System

The proposed automatic sensor monitoring (ASM) system for activity recognition uses a constraintbased ontology to model and detect events based on the sensor data output. 2D video stream data is converted to 3D geometric information, and it is combined with a priori semantic information about the clinical scenario [START_REF] Romdhane | Automatic Video Monitoring System for Assessment of Alzheimer Disease Symptoms[END_REF] . The ontology language is declarative and intuitive (as it uses natural terminology), allowing medical experts to easily define and modify the activities and gait event models (using spatial, temporal, video-tracking and accelerometer data to describe events). For example: "participant [person] is close to the chair [object1]", "participant is sitting [person height less than a given threshold]". Figure4 presents the overall system architecture.

Figure4. Example of sensors based monitoring system architecture using video data [START_REF] Romdhane | Automatic Video Monitoring System for Assessment of Alzheimer Disease Symptoms[END_REF] .

Stride Detection algorithm

A stride detection algorithm was developed for the automatic acquisition of gait information (e.g., walking time, stride length, stride frequency) using a triaxial accelerometer embedded in a wearable device (see Figure 5). The proposed stride detection algorithm chain is organized as follows: data collection, signal pre-processing, and stride detection. At the beginning of the algorithm, the acceleration signals are transmitted to a computer via the RF transceiver. Then a moving average filter is applied to remove high-frequency noises from the raw data. Finally, a threshold method is used to determine the start and end points of the strides.

Statistical Analysis

Results are presented in terms of mean (± standard deviation) for continuous variables. For categorical variables, results are presented in terms of frequency for each modality. Intergroup comparisons for continuous variables used the parametric Student's ttest or the non-parametric Mann-Whitney test (p<0.05) if one of the assumptions of the Student's ttest is not satisfied. Intergroup comparisons for categorical variables (2 modalities) used the Fisher's exact test (p<0.05). All calculations were performed using SPSS software (version 19.0).

RESULTS AND DISCUSSION

RESULTS FOR SCENARIO 01

French experimental site Table 7 shows the results of the French participants performing the activities of the Scenario 01. Activity' parameters were calculated using the human expert annotation about the activities in the video sequences. AD participants needed more time to complete the different physical activities due to a lower displacement speed. The ASM system detects the full set of the guidedactivities scenario (e.g., balance test, repeated sequence of sitting-standing positions) using video data with a detection rate from 96.9 % to 100% (true positive rate).

Table 8 shows the same activities parameters of Table 7, but with activity parameters calculated using the activities detected by the ASM system (36/44 video sequences, where HC=16 and AD=20). Although the absolute values of the parameters calculated using ASM results are different from the parameters values obtained from human annotations, the statistically significant differences between healthy participants and AD patients group were preserved.

Taiwanese experimental site

Table 9 shows the results of Taiwanese participants performing Scenario 01. Activity parameters were calculated based on the annotations of Human expert. Activity parameters of Taiwanese participants agree with the French experimental results in the sense that AD participants took more time to perform the selected activities, probably due to their lower speed of displacement when compared to HC participants.

Results for Scenario 02

French experimental site Tables 10 and11 present participants performance in Scenario 02 at the French experimental site. Table 10 shows global results according to activity parameters: duration spent inside the room to perform the Scenario 02 (seconds) and organizational errors in activity ordering. Activity ordering errors are presented as the number of participants who at least once omitted, repeated, or changed the expected temporal order of activities. Table 11 shows the participants' performance for each activity in terms of speed (seconds), omission, and repetition parameters. The speed term was used instead of activity time duration to imply that lower values of this attribute highlight the ability of a participant at performing the activity faster. AD participants spent more time performing activities that involve sorting or classifying objects (A6 and A7), and they had difficulty to manage the time of reading activity (A1) compared to HC participants.

Taiwanese experimental site

Tables 14 and15 show the mean and standard deviation of the speed of participants' activities and their number of errors at performing the activities proposed in the given order (number of activities skipped or performed in an temporal order different from the expected), respectively. Statistical differences were found in the comparison between AD and HC groups (p<0.01, **; p<0.05, *). Activities S2_A4 (water the plant) and S2_A7c ("ABCD" folder) are significant differences between AD and HC for both activity parameters analyzed (speed and frequency).

Results for Scenario 04

Table 16 shows participants performance for single and dual tasks of Scenario 04. AD subjects needed to spend more time and perform more steps than did the healthy controls to complete both 40m walking tests. Furthermore, AD patients presented a significant shorter stride length and slower stride speed, especially during dual task in the 40m test. In addition, the gait profiles of the AD patients showed lower stride cadence and lower stride frequency compared with those of the healthy controls, but neither phenomena were significant in single and dual tasks.

CONCLUSIONS

Certain similarities are seen between results from the Taiwanese and French sites, although a direct comparison of the results of both sites is not possible due to differences in participant population inclusion criteria. Alzheimer's patients can be characterized by several criteria according to the clinical protocol scenarios designed in SWEET-HOME project. AD participants presented a lower balance and a shorter gait length frequency, as well as an irregular gait cycle. Similar findings were also found by Gillain et al. [START_REF] Gillain | The value of instrumental gait analysis in elderly healthy, MCI or Alzheimer's disease subjects and a comparison with other clinical tests used in single and dual-task conditions[END_REF] who pointed out lower gait speed and lower stride length of AD patients when compared to health controls, in single and in dual tasks (gait speed, stride length, stride cycle frequency, and stride regularity).

Alzheimer's patients have also omitted activities and changed their temporal order indicating a decline in IADLs performance (Scenario 02). Statistically significant differences among AD and HC participants in activity like "watering the plant" could be an indicator of AD participants' difficulty at performing unusual activities.

The proposed automatic sensor monitoring system (ASM and stride detection) provided activity values similar to the ones calculated from events annotated by a human assessor. Although their absolute values differ, they follow the same tendency, and the statistical differences found among AD and HC groups are preserved (Table 8). These findings highlight the use of the proposed approach as a support platform for clinicians to objectively measure AD patients' performance in IADLs and gait analysis. Among the advantages of the ASM system are the stability of its results over time (as it does not suffer from emotional state conditions or biases like stress and fatigue), and its quantitative measurement of patient performance.

FUTURE WORK

Next developments will focus upon automating the detection of activities in the semi-guided and free scenarios of the clinical protocol using the quantitative measurements as evidence for the automated objective assessment of older people frailty. This extended ASM version will be evaluated at differentiating Alzheimer's patients at mild to moderate stages from healthy control participants (comparison between AD/MCI and MCI/Control). A multiple sensor data fusion approach will be also studied to enrich patient activity description, and complement patients motor and cognitive ability monitoring.



  Walking speed test (S1_A5): the assessor asks the participant to walk through the room, following a straight path from one side of the room to another (chair side to video camera side, outward attempt, 4 meters), and then to return (return attempt, 4 meters);  Repeated chair stands testing (S1_A6): The assessor asks the participant to make the first posture transfer (from sitting to standing posture) without using help of his/her arms. The examiner will then ask the participant to repeat the same action five times in a row. Time Up & Go test (TUG -S1_A7): participants start from the sitting position, and at the assessor's signal he/she needs to stand up, to walk a 3 meters path, to make a Uturn in the center of the room, return and sit down again.Scenario 02 (S2) or semi-directed activities (indoor) aims at evaluating the degree of independence of the older people by performing IADLs in a given order. The participant stays alone in the room with a list of activities to perform, and he/she has to leave the room only when feeling that has completed the required tasks (no maximum time). It includes 7 indoor IADLs to be performed in the following order:  Reading for 2 minutes (S2_A1);  Warming water (S2_A2);  Making a call for phone number 34775 (S2_A3);  Watering a plant (S2_A4);  Watching TV (S2_A5);  Classifying cards by color (S2_A6);  Taking ABCD Folder in the zone Office (S2_A7a); Matching ABCD letters of the ABCD Folder with ABCD letters placed over the room (S2_A7b), Returning ABCD folder to its place (S2_A7c), and;  Leaving the room (S2_A8).

Figure2.

  Figure2. French indoor experimental room, observation room

Figure 5 .

 5 Figure 5. Proposed stride detection device

Figure 6

 6 Figure 6 and 7 show the time taken by the Taiwanese cohort to perform the right and left leg activities of balance test, respectively. Significant differences were found in both tests in the comparison of AD and HC groups (p<0.01 for right leg standing and left leg standing).

Figure 6 .

 6 Figure 6. Time taken by Participants to perform the activity "right leg standing" of the balance test in the Taiwanese clinical trial

Figure7.

  Figure7. Time taken by the Participants to perform the activity "left leg standing" of the balance test in Taiwanese clinical trial

Figure8.Figure 9 .

 9 Figure8. Mean speed of French participants in the TUG test

Table 1 .

 1 French participants on S1

		HC (N=21)	AD (N=23)
	Female, N (%)	12 (50%)	16(68.75%)
	Age, years(mean ± SD) 73.4±6.09	76.7±7.84
	MMSE(mean ± SD)	28.4±0.98	21.35±3.97

Table 2 .

 2

French participants on S2 HC (N=10) AD (N=16) Female, N (%) 5 (50%) 11(68.8%) Age, years (mean±SD) 73.9±6.24 76.7± 7.56 MMSE (mean ± SD) 28.1±1.85 20.7± 3.70

Table 3 .

 3 Taiwanese

	participants on S1 and S2
		HC (N=45)	AD (N=36)
	Female, N (%)	24 (53.3%)	21 (58.3%)
	Age,years (mean ± SD) 64.51±8,33	70.25±9.25
	MMSE (mean ± SD)	27.60±2.04	23.44±3.32
	Table 4.Taiwanese participants on S4
		HC(N=17)	AD(N=16)
	Female, N (%)	9 (52.9%)	9 (56.3%)

Age,years (mean ± SD) 62.35±2.04 66.69±5.69 MMSE (mean ± SD) 27.65±2.40 24.19±3.62

Table 5 .

 5 Sensors used for older people activity anal-

	ysis within each scenario
		France	Taiwan
		2D video camera	2D video camera
	S1	Automatic Sensor Moni-(Activities annotation)	(Activities annotation)
		toring
	S2	2D video camera Activities annotation	2D video camera Activities annotation
	S4	-	Tri-axial accelerometer

Table 7 .

 7 French participants' performance for Scenario 01 calculated using annotations of human expert

		HC	AD	p-value
	S1_A5
	-Walking speed,
	(Go attempt) (m/s)	0.88±0.25	0.69±0.20	0.009 **
	-Walking speed	1.02±0.21	0.77±0.18	< 0.001
	(Go back attempt)			**
	(m/s)
	S1_A6
	-Duration (s)	14.90±5.75	19.7±6.79	0.012 *
	-Duration/ number	1.50±0.58	1.9±0.63	0.006 **
	of transfers
	S1_A7 -Duration (s)	10.30±4.12	14.6±6.16	0.002 **
	P-values were computed using Student's t-test for the
	parameters of S1_A5, and by using Wilcoxon Test for the
	parameters of S1_A6 and S1_A7; (*) Statistical signifi-
	cance at p < 0.05; (**) Statistical significance at p < 0.01

Table 8

 8

	. French participants' performance for Sce-
	nario 01 calculated using the ASM system results
		HC	AD	p-value
		(N=16)	(N=20)
	S1_A5
	-Walking speed, (Go	1.06±0.23	0.79±0.23	0.001**
	attempt) (m/s)
	-Walking speed (Go	1.20±0.31	0.89±0.23	0.002**
	back attempt) (m/s)
	S1_A6
	-Duration (s)	12.8±5.40	17.7±6.31	0.006**
	-Duration/number of	1.3±0.53	1.7±0.56	0.002**
	transfers
	S1_A7
	-Duration (s)	8.8±3.80	12.1±5.64.	0.007*
	P-values were computed by using Student's t-test for the on
	parameters of S1_A5, and by using Wilcoxon Test for the
	parameters of S1_A6 and S1_A7; (*) Statistical significance at
	p<0.05; (**) Statistical significance at p<0.01

Table 9 .

 9 Taiwanese participants' performance for Scenario 01.

			HC	AD	p-value
	S1_A5
	-Walking		0.38 ± 0.08	0.32 ± 0.74	0.001**
	speed, (Go
	attempt)		0.41 ± 0.07	0.34 ± 0.08	<0.001**
	(m/s)
	-Walking
	speed (Go
	back	at-
	tempt) (m/s)
	S1_A6
	-Duration (s)	14.5 ± 3.33	20.0 ± 9.41	0.001**
	-Duration/		1.4 ± 0.33	2.00 ± 0.94	0.001**
	number	of
	transfers
	S1_A7		10.2 ± 2.42	13.4 ± 3.34	<0.001*
	-Duration (s)

P-values were computed using Student's t-test for parameters of S1_A5, and using Wilcoxon Test for the parameters of S1_A6 and S1_A7; (*) Statistical significance at p<0.05; (**) Statistical significance at p<0.01.

Table 10 .

 10 Global performance of the French participants for Scenario 02

	Activity	HC	AD	p-value
	-Total time
	spent in the	454±160.4	715±352	0.060
	room (s)
	Number of Participants presenting errors about:
	-Activity omis-sion (n, %)	0 (0%)	2 (12.5%)	0.508
	-Activity repeti-tion (n,%)	0 (0%)	6 (37.5%)	0.053

Table 11 .

 11 French participants performance for each activity of Scenario 02

	Activity	HC	AD	p-value
	S2_A1
	-Speed (s)	138±79.6	88±215.2	0.001 (**)
	-Omitted, N(%)	0 (0%)	0 (0%)	X
	-Repeated, N(%)	0 (0%)	2 (12.5%)	0.508
	S2_A2
	-Speed (s)	4±3.8	8±13.4	0.660
	-Omitted, N(%)	0 (0%)	0 (0%)	X
	-Repeated, N(%)	0 (0%)	2 (12.5%)	0.508
	S2_A3
	-Speed (s)	25±12.3	28±20.3	0.979
	-Omitted, N(%)	0 (0%)	0 (0%)	X
	-Repeated, N(%)	0 (0%)	2 (12.5%)	0.508
	S2_A4
	-Speed (s)	9±4.4	11±7.2	0.856
	-Omitted, N(%)	0 (0%)	0 (0%)	X
	-Repeated, N(%)	0 (0%)	3 (18.75%)	0.262
	S2_A5
	-Speed (s)	32±24.3	57±57.9	0.165
	-Omitted, N(%)	0 (0%)	0 (0%)	X
	-Repeated, N(%)	0 (0%)	2 (12.5%)	0.508
	S2_A6
	-Speed (s)	78±30.9	143±176	0.216
	-Omitted, N(%)	0 (0%)	1 (6.25%)	1.00
	-Repeated, N(%)	0 (0%)	3 (18.75%)	0.262
	S2_A7
	-Speed (s)	49±21.3	80±48.5	0.129
	-Omitted, N(%)	0 (0%)	1 (6.25%)	1.00
	-Repeated, N(%)	0 (0%)	0 (0%)	x
	P-values for continuous variables were computed using
	Wilcoxon test; p-values for categorical variables (2 modali-
	ties) were computed using Fisher's exact test; (*) Statistical
	significance at p<0.05; (**) Statistical significance at
	p<0.01.
	Tables 12 and 13 present examples of an AD patient
	and a HC participant performance in Scenario 02
	activities in the French experimental site. In this
	example the AD participant forgot to perform 3 activi-

Table 14

 14

		. Mean and standard deviation of partici-
	pants speed in activities of Scenario 02
			Speed (seconds)
	Act. A1	A2 *	A3 **
	AD	129.8±57.8	41.4±21.0	39.7±22.5
	HC	121.1±49.2	23.8±12.1	21.7±6.9
	Act. A4 **	A5	A6
	AD	52.3±28.7	36.4±24.4	98.8±35.5
	HC	25.3±10.4	23.0±11.7	69.5±21.0
	Act. A7a *	A7b *	A7c **
	AD	19.4±21.2	78.3±37.9	87.0±46.3
	HC	9.4±7.4	50.4±21.0	50.8±19.7
	Act. A8 *
	AD	48.1±35.1
	HC	23.2±10.9

Table 16 .

 16 Performance of participants for single and dual task of Scenario 04

	Tasks	Single		Dual
	Participants	AD	HC	AD	HC
	Times**	34.24±	28.40±3.	64.03±30.	37.74±7.
		5.24	45	77	58
	No. of	32.25±	28.18±2.	37.81±7.2	28.76±2.
	Stride**	3.36	79	2	49
	Stride	57.01±	59.87±4.	39.80±11.	47.15±8.
	Cadence	4.65	79	34	33
	(stride/min)
	Stride	1.25±0	1.43±0.1	1.09±0.20 1.38±0.1
	Length (m)	.12	4		1
	**
	Stride	0.95±0	1.00±0.1	0.75±0.15 0.83±0.1
	Frequency	.08	0		1
	(Hz)
	Stride	1.20±0	1.44±0.2	0.83±0.26 1.14±0.1
	Speed	.18	0		9
	(m/s) **
	(**: p<0.01)


Keywords: health & self-esteem, multiple sensors monitoring systems, Alzheimer's dementia

Older people population is expected to grow dramatically over the next 20 years (including Alzheimer's patients), while the number of people able to provide care will decrease. We present the development of medical and information and communication technologies to support the diagnosis and evaluation of dementia progress in early stage Alzheimer disease (AD) patients. Method: We compared video and accelerometers activity assessment for the estimation of older people performance in instrumental activities of daily living (IADL) and physical tests in the clinical protocol developed by the Memory Center of the Nice Hospital and the Department of Neurology at National Cheng Kung University Hospital -Taiwan. This clinical protocol defines a set of IADLs (e.g., preparing coffee, watching TV) that could provide objective information about dementia symptoms and be realistically achieved in the two sites observation room. Previous works studied accelerometers activity assessment for the detection of changes in older people gait patterns caused by dementia progress, or video-based event detection for personal self-care activities (ADLs)[1, 2, 3], but none has used both sensors for IADLs analysis. The proposed system uses a constraint-based ontology to model and detect events based on different sensors readings (e.g., 2D video stream data is converted to 3D geometric information that is combined with a priori semantic information, like defined spatial zones or posture estimations given by accelerometer). The ontology language is declarative and intuitive (as it uses natural terminology), allowing medical experts to define and modify the IADL models. The proposed system was tested with 44 participants (healthy=21, AD=23). A stride detection algorithm was developed by the Taiwanese team for the automatic acquisition of patients gait parameters (e.g., stride length, stride frequency) using a tri-axial accelerometer embedded in a wearable device. It was tested with 33 participants (healthy=17, Alzheimer = 16) during a 40 meters walking test. Results & Discussion: The proposed system detected the full set of activities of the first part of our clinical protocol (e.g., repeated transfer test, walking test) with a true positive rate of 96.9 % to 100%. Extracted gait parameters and automatically detected IADLs will be future analyzed for the evaluation of differences between Alzheimer patients at mild to moderate stages and healthy control participants, and for the monitoring of patients motor and cognitive abilities.


Guillermo Moncecchi

Jean-Luc Minel

Improving Speculative Language Detection using Linguistic Knowledge

In this paper we present an iterative method-ology to improve classifier performance by in-corporating linguistic knowledge, and propose a way to incorporate domain rules into the learning process. We applied the methodol-ogy to the tasks of hedge cue recognition and scope detection and obtained competitive re-sults on a publicly available corpus.

Introduction

A common task in Natural Language Processing (NLP) is to extract or infer factual information from textual data. In the field of natural sciences this task turns out to be of particular importance, because science aims to discover or describe facts from the world around us. Extracting these facts from the huge and constantly growing body of research articles in areas such as, for example, molecular biology, becomes increasingly necessary, and has been the subject of intense research in the last decade [START_REF] Ananiadou | Text mining and its potential applications in systems biology[END_REF]. The fields of information extraction and text mining have paid particular attention to this issue, seeking to automatically populate structured databases with data extracted or inferred from text. In both cases, the problem of speculative language detection is a challenging one, because it may correspond to a subjective attitude of the writer towards the truth value of certain facts, and that information should not be lost when the fact is extracted or inferred.

When researchers express facts and relations in their research articles, they often use speculative lan-guage to convey their attitude to the truth of what is said. Hedging, a term first introduced by [START_REF] Lakoff | Hedges: A study in meaning criteria and the logic of fuzzy concepts[END_REF] to describe 'words whose job is to make things fuzzier or less fuzzy' is 'the expression of tentativeness and possibility in language use' [START_REF] Hyland | The author in the text: Hedging scientific writing[END_REF], and is extensively used in scientific writing. Hyland (1996a) reports one hedge in every 50 words of a corpus of research articles; [START_REF] Light | The language of bioscience: Facts, speculations, and statements in between[END_REF] mention that 11% of the sentences in MEDLINE contain speculative language. [START_REF] Vincze | The bioscope corpus: biomedical texts annotated for uncertainty, nega-tion and their scopes[END_REF] report that 18% of the sentences in the scientific abstracts section of the Bioscope corpus correspond to speculations.

Early work on speculative language detection tried to classify a sentence either as speculative or non-speculative (see, for example, [START_REF] Medlock | Weakly supervised learning for hedge classification in scientific literature[END_REF]). This approach does not take into account the fact that hedging usually affects propositions or claims [START_REF] Hyland | The author in the text: Hedging scientific writing[END_REF] and that sentences often include more than one of them. When the Bioscope corpus [START_REF] Vincze | The bioscope corpus: biomedical texts annotated for uncertainty, nega-tion and their scopes[END_REF] was developed the notions of hedge cue (corresponding to what was previously called just 'hedges' in the literature) and scope (the propositions affected by the hedge cues) were introduced. In this context, speculative language recognition can be seen as a two-phase process: first, the existence of a hedge cue in a sentence is detected, and second, the scope of the induced hedge is determined. This approach was first used by [START_REF] Morante | Learning the scope of negation in biomedical texts[END_REF] and subsequently in many of the studies presented in the CoNLL-2010 Conference Shared Task (Farkas et al., 2010a), and is the one used in this paper. For example, the sentence

(1) This finding { suggests suggests that the BZLF1

promoter { may may be regulated by the degree of squamous differentiation} may } suggests .

contains the word 'may' that acts as a hedge cue (i.e. attenuating the affirmation); this hedge only affects the propositions included in the subordinate clause that contains it. Each of these phases can be modelled (albeit with some differences, described in the following sections) as a sequential classification task, using a similar approach to that commonly used for named entity recognition or semantic labelling: every word in the sentence is assigned a class, identifying spans of text (as, for example, scopes) with, for example, a special class for the first and last element of the span.

Correctly learning these classes is the computational task to be solved.

In this paper we present a methodology and machine learning system implementing it that, based on previous work on speculation detection, studies how to improve recognition by analysing learning errors and incorporating advice from domain experts in order to solve the errors without hurting overall performance. The methodology proposes the use of domain knowledge rules that suggest a class for an instance, and shows how to incorporate them into the learning process. In our particular task domain knowledge is linguistic knowledge, as hedging and scopes issues are general linguistic devices. In this paper we are going both terms interchangeably. The paper is organized as follows. In Section 2 we review previous theoretical work on speculative language and the main computational approaches to the task of detecting speculative sentences. Section 3 briefly describes the corpus used for training and evaluation. In Section 4 we present the specific computational task to which our methodology was applied. In Section 5 we present the learning methodology we propose to use, and describe the system we implemented, including lexical, syntactic and semantic attributes we experimented with. We present and discuss the results obtained in Section 6. Finally, in Section 7 we analyse the approach presented here and discuss its advantages and problems, suggesting future lines of research.

Related work

The grammatical phenomenon of modality, defined as 'a category of linguistic meaning having to do with the expression of possibility and necessity' [START_REF] Kail Von | Modality and Language[END_REF] Although hedging is a concept that is closely related to epistemic modality, they are different: modality is a grammatical category, whereas hedging is a pragmatic position [START_REF] Morante | Modality and negation: An introduction to the special issue[END_REF]. This phenomenon has been theoretically studied in different domains and particularly in scientific writing [START_REF] Hyland | The author in the text: Hedging scientific writing[END_REF]Hyland, 1996b;Hyland, 1996a).

From a computational point of view, speculative language detection is an emerging area of research, and it is only in the last five years that a relatively large body of work has been produced. In the remainder of this section, we survey the main approaches to hedge recognition, particularly in English and in research discourse. [START_REF] Medlock | Weakly supervised learning for hedge classification in scientific literature[END_REF] applied a weakly supervised learning algorithm to classify sentences as speculative or non-speculative, using a corpus they built and made publicly available. [START_REF] Morante | Learning the scope of hedge cues in biomedical texts[END_REF] not only tried to detect hedge cues but also to identify their scope, using a metalearning approach based on three supervised learning methods. They achieved an F1 of 84.77 for hedge identification, and 78.54 for scope detection (using gold-standard hedge signals) in the Abstracts sections of the Bioscope corpus. Task 2 of the CoNLL-2010 Conference Shared Task (Farkas et al., 2010b) proposed solving the problem of in-sentence hedge cue phrase identi-fication and scope detection in two different domains (biological publications and Wikipedia articles), based on manually annotated corpora. The evaluation criterion was in terms of precision, recall and F-measure, accepting a scope as correctly classified if the hedge cue and scope boundaries were both correctly identified. The best result on hedge cue identification [START_REF] Tang | A cascade method for detect-ing hedges and their scope in natural language text[END_REF] obtained an F-score of 81.3 using a supervised sequential learning algorithm to learn BIO classes from lexical and shallow parsing information, also including certain linguistic rules. For scope detection, [START_REF] Morante | Memory-based resolution of in-sentence scopes of hedge cues[END_REF] obtained an F-score of 57.3, using also a sequence classification approach for detecting boundaries (tagged in FOL format, where the first token of the span is marked with an F, while the last one is marked with an L). The attributes used included lexical information, dependency parsing information, and some features based on the information in the parse tree.

The approximation of [START_REF] Velldal | Resolving speculation: Maxent cue classification and dependency-based scope rules[END_REF] for scope detection was somewhat different: they de-veloped a set of handcrafted rules, based on depen-dency parsing and lexical features. With this ap-proach, they achieved an F-score of 55.3, the third best for the task. Similarly, [START_REF] Kilicoglu | A highprecision approach to detecting hedges and their scopes[END_REF] used a pure rule-based approach based on constituent parse trees in addition to syntactic dependency relations, and achieved the fourth best Fscore for scope detection, and the highest precision of the whole task (62.5). In a recent paper, Velldal et al. ( 2012) reported a better F-score of 59.4 on the same corpus for scope detection using a hybrid approach that combined a set of rules on syntactic features and n-gram features of surface forms and lexical information and a machine learning system that selected subtrees in constituent structures.

Corpus

The system presented in this paper uses the Bioscope corpus [START_REF] Vincze | The bioscope corpus: biomedical texts annotated for uncertainty, nega-tion and their scopes[END_REF] as a learning source and for evaluation purposes. The Bioscope corpus is a freely available corpus of medical free texts, biological full papers and biological abstracts, annotated at a token level with negative and speculative keywords, and at sentence level with their linguistic scope. 2008), gives some statistics related to hedge cues and sentences for the three sub corpora included in Bioscope.

For the present study, we usee only the Abstract sub corpus for training and evaluation. We randomly separated 20% of the corpus, leaving it for evaluation purposes. We further sub-divided the remaining training corpus, separating another 20% that was used as a held out corpus. All the models presented here were trained on the resulting training corpus and their performance evaluated on the held out corpus. The final results were computed on the previously unseen evaluation corpus.

Task description

From a computational point of view, both hedge cue identification and scope detection can be seen as a sequence classification problem: given a sentence, classify each token as part of a hedge cue (or scope) or not. In almost every classification problem, two main approaches can be taken (although many variations and combinations exist in the literature): build the classifier as a set of handcrafted rules, which, from certain attributes of the instances, decide which category it belongs to, or learn the classifier from previously annotated examples, in a supervised learning approach. The rules approach is particularly suitable when domain experts are available to write the rules, and when features directly represent linguistic information (for example, POS-tags) or other types of domain information. It is usually a time-consuming task, but it probably grasps the subtleties of the linguistic phenomena studied better, making it possible to take them into account when building the classifier. The supervised learning approach needs tagged data; in recent years the availability of tagged text has grown, and this type of method has become the state-of-the-art solution for many NLP problems.

In our particular problem, we have both tagged data and expert knowledge (represented by the body of work on modality and hedging), so it seems reasonable to see how we can combine the two methods to achieve better classification performance.

Identifying hedge cues

The best results so far for this task used a token classification approach or sequential labelling techniques, as Farkas et al. (2010b) note. In both cases, every token in the sentence is assigned a class label indicating whether or not that word is acting as a hedge cue. To allow for multiple-token hedge cues, we identify the first token of the span with the class B and every other token in the span with I, keeping the O class for every token not included in the span, as the following example shows:

( After token labelling, hedge cue identification can be seen as the problem of assigning the correct class to each token of an unlabelled sentence. Hedge cue identification is a sequential classification task: we want to assign classes to an entire ordered sequence of tokens and try to maximize the probability of assigning the correct classes to every token in the sequence, considering the sequence as a whole, not just as a set of isolated tokens.

Determining the scope of hedge cues

The second sub-task involves marking the part of the sentence affected by the previously identified hedge cue. Scopes are also spans of text (typically longer than multi-word hedge cues), so we could use the same reduction to a token classification task. Being longer, FOL classes are usually used for classification, identifying the first token of the scope as F, the last token as L and any other token in the sentence as O. Scope detection poses an additional problem: hedge cues cannot be nested, but scopes (as we have already seen) usually are. In example 1, the scope of 'may' is nested within the scope of 'suggests'. To overcome this, [START_REF] Morante | Learning the scope of hedge cues in biomedical texts[END_REF] propose to generate a different learning example for each cue in the sen-tence. In this setting, each example becomes a pair hlabelled sentence, hedge cue positioni. So, for example 1, the scope learning instances would be:

( Learning on these instances, and using a similar approach to the one used in the previous task, we should be able to identify scopes for previously unseen examples. Of course, the two tasks are not independent: the success of the second one depends on the success of the first. Accordingly, evaluation of the second task can be done using gold standard hedge cues or with the hedge cues learned in the first task.

Methodology and System Description

To approach both sequential learning tasks, we follow a learning methodology (depicted in Figure 1), that starts with an initial guess of attributes for supervised learning and a learning method, and tries to improve its performance by incorporating domain knowledge. We consider that expressing this knowledge through rules (instead of learning features) is a better way for a domain expert to suggest new useful information or to generalize certain relations between attributes and classification results when the learning method cannot achieve this because of insufficient training data. These rules, of course, have to be converted to attributes to incorporate them into the learning process. These attributes are what we call knowledge rules and their generation will be described in the Analysis section.

Preprocessing

Before learning, we propose to add every possible item of external information to the corpus so as to integrate different sources of knowledge (either the result of external analysis or in the form of semantic resources). After this step, all the information is consolidated into a single structure, facilitating subsequent analysis. In our case, we incorporate POS-tagging information, resulting from the application of the GENIA tagger [START_REF] Tsuruoka | Developing a robust Part-of-Speech tagger for biomedical text[END_REF], and deep syntax information obtained with the application of the Stanford Parser [START_REF] Klein | Accurate unlexicalized parsing[END_REF], leading to a syntax-oriented representation of the training data. For a detailed description of the enriching process, the reader is referred to [START_REF] Moncecchi | Enriching the bioscope corpus with lexical and syntactic information[END_REF].

Initial Classifier

The first step for improving performance is, of course, to select an initial set of learning features, and learn from training data to obtain the first classifier, in a traditional supervised learning scenario.

The sequential classification method will depend on the addressed task. After learning, the classifier is applied on the held out corpus to evaluate its performance (usually in terms of Precision, Recall and F1-measure), yielding performance results and a list of errors for analysis. This information is the source for subsequent linguistic analysis. As such, it seems important to provide ways to easily analyse instance attributes and learning errors. For our tasks, we have developed visualization tools to inspect the tree representation of the corpus data, the learning attributes, and the original and predicted classes.

Analysis

From the classifier results on the held-out corpus, an analysis phase starts, which tries to incorporate linguistic knowledge to improve performance. One typical form of introducing new information is through learning features: for example, we can add a new attribute indicating if the current instance (in our case, a sentence token) belongs to a list of common hedge cues. However, linguistic or domain knowledge can also naturally be stated as rules that suggest the class or list of classes that should be assigned to instances, based on certain conditions on features, linguistic knowledge or data observation. For example, based on corpus annotation guidelines, a rule could state that the scope of a verb hedge cue should be the verb phrase that includes the cue, as in the expression

(5) This finding { suggests suggests that the BZLF1 promoter may be regulated by the degree of squamous differentiation} suggests .

We assume that these rules take the form 'if a condition C holds then classify instance X with class Y'.

In the previous example, assuming a FOL format for scope identification, the token 'suggest' should be assigned class F and the token 'differentiation' should be assigned class L, assigning class O to every other token in the sentence.

The general problem with these rules is that as we do not know in fact if they always apply, we do not want to directly modify the classification results, but to incorporate them as attributes for the learning task. To do this, we propose to use a similar approach to the one used by Rosa ´ (2011), i.e. to incorporate these rules as a new attribute, valued with the class predictions of the rule, trying to 'help' the classifier to detect those cases where the rule should fire, without ignoring the remaining attributes. In the previous example, this attribute would be (when the rule condition holds) valued F or L if the token corresponds to the first or last word of the enclosing verb phrase, respectively. We have called these attributes knowledge rules to reflect the fact that they suggest a classification result based on domain knowledge. This configuration allows us to incorporate heuristic rules without caring too much about their potential precision or recall ability: we expect the classification method to do this for us, detecting correlations between the rule result (and the rest of the attributes) and the predicted class.

There are some cases where we do actually want to overwrite classifier results: this is the case when we know the classifier has made an error, because the results are not well-formed. For example, we have included a rule that modifies the assigned classes when the classifier has not exactly found one F token and one L token, as we know for sure that something has gone wrong. In this case, we decided to assign the scope based on a series of postprocessing rules: for example, assign the scope of the enclosing clause in the syntax tree as hedge scope, in the case of verb hedge cues.

For sequential classification tasks, there is an additional issue: sometimes the knowledge rule indicates the beginning of the sequence, and its end can be determined using the remaining attributes. For example, suppose the classifier suggests the class

O O O O O O F O O O O O O O L O O O O O F O O O O O O O O O O L O O O O O F O O O O O O O O O O O O
Table 2: Evaluation instance where the scope ending could not be identified scope in the learning instance shown in table 2 (using as attributes the scopes of the parent and grandparent constituents for the hedge cue in the syntax tree). If we could associate the F class suggested by the classifier with the grand parent scope rule, we would not be concerned about the prediction for the last token, because we would knew it would always correspond to the last token of the grand parent clause. To achieve this, we modified the class we want to learn, introducing a new class, say X, instead of F, to indicate that, in those cases, the L token must not be learned, but calculated in the postprocessing step, in terms of other attributes' values (in this example, using the hedge cue grandparent constituent limits). This change also affects the classes of training data instances (in the example, every training instance where the scope coincides with the grand parent scope attribute will have its F-classified token class changed to X). In the previous example, if the classifier assigns class X to the 'the' token, the postprocessing step will change the class assigned to the 'differentiation' token to L, no matter which class the classifier had predicted, changing also the X class to the original F, yielding a correctly identified scope. After adding the new attributes and changing the relevant class values in the training set, the process starts over again. If performance on the held out corpus improves, these attributes are added to the best configuration so far, and used as the starting point for a new analysis. When no further improvement can be achieved, the process ends, yielding the best We applied the proposed methodology to the tasks of hedge cue detection and scope resolution. We were mainly interested in evaluating whether systematically applying the methodology would indeed improve classifier performance. The following sections show how we tackled each task, and how we managed to incorporate expert knowledge and improve classification.

Hedge Cue Identification

To identify hedge cues we started with a sequential classifier based on Conditional Random Fields [START_REF] Lafferty | Conditional random fields: Probabilistic mod-els for segmenting and labeling sequence data[END_REF], the state-of-the-art classification method used for sequence supervised learning in many NLP tasks. The baseline configuration we started with included a size-2 window of surface forms to the left and right of the current token, pairs and triples of previous/current surface forms. This led to a highly precise classifier (an F-measure of 95.5 on the held out corpus). After a grid search on different configurations of surface forms, lemmas and POS tags, we found (somewhat surprisingly) that the best precision/recall tradeoff was obtained just using a window of size 2 of unigrams of surface forms, lemmas and tokens with a slightly worse precision than the baseline classifier, but compen- Table 3: Classification performance on the held out corpus for hedge cue detection. Conf1 corresponds to windows of Word, Lemma and POS attributes and Conf2 incorporates hedge cue candidates and cooccuring words sated by an improvement of about six points in recall, achieving an F-score of 86.9. In the analysis step of the methodology we found that most errors came from False Negatives, i.e. words incorrectly not marked as hedges. We also found that those words actually occurred in the training corpus as hedge cues, so we decided to add new rule attributes indicating membership to certain semantic classes. After checking the literature, we added three attributes:

• Hyland words membership: this feature was set to Y if the word was part of the list of words identified by [START_REF] Hyland | Metadiscourse: Exploring Interaction in Writing[END_REF] • Hedge cue candidates: this feature was set to Y if the word appeared as a hedge cue in the training corpus

• Words co-occurring with hedge cue candidates: this feature was set to Y if the word cooccured with a hedge cue candidate in the training cor-pus. This feature is based on the observation that 43% of the hedges in a corpus of scientific articles occur in the same sentence as at least another device [START_REF] Hyland | The author in the text: Hedging scientific writing[END_REF].

After adding these attributes and tuning the window sizes, performance improved to an F-score of 87.5 in the held-out corpus

Scope identification

To learn scope boundaries, we started with a similar configuration of a CRF classifier, using a window of size 2 of surface forms, lemmas and POS-tags, and the hedge cue identification attribute (either obtained from the training corpus when using gold standard hedge cues or learned in the previous step), achieving a performance of 63.7 in terms of F-measure. When we incorporated information in the form of a knowledge rule that suggested the scope of the constituent of the parsing tree headed by the parent node of the first word of the hedge cue, and an attribute containing the parent POS-tag, performance rapidly improved about two points measured in terms of Fscore. After several iterations, and analyzing classification errors, we included several knowledge rules, attributes and postprocessing rules that dramatically improved performance on the held-out corpus:

• We included attributes for the scope of the next three ancestors of the first word of the hedge cue in the parsing tree, and their respective POStags, in a similar way as with the parent. We also included a trigram with the ancestors POS from the word upward in the tree.

• For parent and grandparent scopes, we incorporated X and Y classes instead of F, and mod-ified postprocessing to use the last token of the corresponding scope when one of these classes was learned.

• We modified the ancestors scopes to reflect some corpus annotation guidelines or other cri-teria induced after data examination. For ex-ample, we decided not to include adverbial phrases or prepositional phrases at the begin-ning of scopes, when they corresponded to a clause, as in (6) In addition,{unwanted and potentially hazardous specificities may be elicited. . . }

• We added postprocessing rules to cope with cases where (probably due to insufficient train-ing data), the classifier missclasified certain in-stances. For example, we forced classification to use the next enclosing clause (instead of verb phrase), when the hedge cue was a verb conju-gated in passive voice, as in (7) {GATA3 , a member of the GATA family that is abundantly expressed in the T-lymphocyte lineage , is thought to participate in ...}. • We excluded references at the end of sentences from all the calculated scopes.

• We forced classification to the next S,VP or NP ancestor constituent in the syntax tree (depend-ing on the hedge cue POS), when full scopes could not be determined by the statistical clas-sifier (missing either L or F, or learning more than one of them in the same sentence).

Table 4 summarizes the results of scope identification in the held out corpus. The first results were obtained using gold-standard hedge cues, while the second ones used the hedge cues learned in the previous step (for hedge cue identification, we used the best configuration we found). In the gold-standard results, Precision, Recall and the F-measure are the same because every False Positive (incorrectly marked scope) implied a False Negative (the missed right scope).

Evaluation

To determine classifier performance, we evaluated the classifiers found after improvement on the evaluation corpus. We also evaluated the less efficient classifiers to see whether applying the iterative improvement had overfitted the classifier to the corpus. To evaluate scope detection, we used the best configuration found in the evaluation corpus for hedge cue identification. Tables 5 and6 show the results for the hedge cue recognition and scope resolution, respectively. In both tasks, classifier performance improved in a similar way to the results obtained on the held out corpus. Finally, to compare our results with state-of-the-art methods (even though that was not the main objective of the study), we used the corpus of de CoNLL 2010 Shared Task to train and evaluate our classifiers, using the best configurations found in the evaluation corpus, and obtained competitive results in both subtasks of Task 2. Our classifier for hedge cue detection achieved an F-measure of 79.9, better than the third position in the Shared Task for hedge identification. Scope detection results (using learned hedge cues) achieved an F-measure of 54.7, performing better than the fifth result in the corresponding task, and five points below the best results obtained so far in the corpus (Velldal et

Conclusions and Future Research

In this paper we have presented an iterative methodology to improve classifier performance by incorporating linguistic knowledge, and proposed a way to incorporate domain rules to the learning process. We applied the methodology to the task of hedge cue recognition and scope finding, improving performance by incorporating information of training corpus occurrences and co-occurrences for the first task, and syntax constituents information for the second. In both tasks, results were competitive with the best results obtained so far on a publicly available corpus. This methodology could be easily used for other sequential (or even traditional) classification tasks. Two directions are planned for future research: first, to improve the classifier results by incorporat-ing more knowledge rules such as those described by [START_REF] Velldal | Speculation and negation: Rules, rankers, and the role of syntax[END_REF] or semantic resources, specially for the scope detection task. Second, to improve the methodology, for example by adding some way to select the most common errors in the held out corpus and write rules based on their examination.

Figure 1 :

 1 Figure 1: Methodology overview

Table 1 :

 1 Bioscope corpus statistics about hedging Table 1, extracted from Vincze et al. (

		Clinical Full Abstract
	#Documents	954	9	1273
	#Sentences	6383	2670	11871
	%Hedge Sentences	13.4	19.4	17.7
	#Hedge cues	1189	714	2769

Table 5 :

 5 Classification performance on the evaluation corpus for hedge cue detection

	Configuration Gold-P P	R	F1
	Baseline	74.0	71.9	68.1	70.0
	Conf1	76.5	74.4	70.2	72.3
	Conf2	80.0	77.2	72.9	75.0
	Conf3	83.1	80.0	75.2	77.3
	Conf4	84.7	80.1	75.8	77.9

Table 6 :

 6 Classification performance on the evaluation corpus for scope detection

Table 7 :

 7 al., Classification performance compared with best results in CoNLL Shared Task. Figures represent Precision/Recall/F1-measure 2012). Table 7 summarizes these results in terms of Precision/Recall/F1-measure.

		Hedge cue iden-	Scope detection
		tification
	Best results	81.7/81.0/81.3	59.6/55.2/57.3
	Our results	83.2/76.8/79.9	56.7/52.8/54.7