William Billingsley

Cindy Gallois

Andrew Smith

Timothy Marks

Fernando Bernal

Marcus Watson
email: marcus.watson@nicta.com.au

{william Billingsley

Towards a Diagnostic Toolbox for Medical Communication

Keywords: Communication, Visualisation, Patient History Taking, Natural Language Processing, Artificial Intelligence

Poor communication is a major cause of adverse patient events in hospitals. Although sophisticated simulators are in use for performing medical operations, there is comparatively little technology support being used for improving communication skills including patient history taking. Artificial Intelligence and Natural Language Processing researchers have developed sophisticated algorithms for analysing conversations. We are experimentally developing software that can visualise the combined output of these algorithms, as a diagnostic toolkit for medical communication.

Introduction

Adverse patient events occur in between 3.7% [START_REF] Brennan | Incidence of adverse events and negligence in hospitalized patients: results of the Harvard medical practice study[END_REF] and 16.6% [START_REF] Wilson | The quality in Australian healthcare survey[END_REF] of hospitalisations.

Even the studies that have found the lower rates have concluded that a substantial amount of patient harm happens through medical management. Failures of communication have been found to be a major contributor in up to forty per cent of these events [START_REF]Sentinel events in Australian public hospitals 2004-05[END_REF], and seventy per cent of those that cause serious harm [START_REF] Leonard | The human factor: the critical importance of effective teamwork and communication in providing safe care[END_REF]. One Australian study attributed communication as a major cause to approximately 14,000 preventable deaths each year [START_REF] Zinn | Preventable deaths in Australian hospitals[END_REF]. If failures of communication are such a large contributor to patients being harmed or killed within the health service, then this suggests that improving medical professionals' skills and habits in communication would make a significant improvement to patient outcomes. Technology now plays a pivotal role in diagnosing patient health issues. Particularly, visualisations of technical inspections of the body -such as ultrasound, electrocardiograms, and magnetic resonance imagery -have become a day-to-day part of diagnostic practice. We propose that technical inspections of medical communication could similarly help to diagnose and understand how it fails.

There are, of course, many different modes of communication in the health service. A failure of communication could include, for instance, that something was written incorrectly on a patient chart, that a nurse was not informed of relevant information that was known to a doctor, or that a misunderstanding took place during a medical conversation. It is very unlikely that a single technology could be designed that could encompass all of these different kinds of communication and the ways they can fail. Instead, we propose starting with a kind of communication event that every patient undergoes, and that precedes every other medical interaction: patient history taking.

Background

Many research projects have analysed medical communication [START_REF] Roter | The Roter interaction analysis system (RIAS): utility and flexibility for analysis of medical interactions[END_REF][START_REF] Apker | Exploring emergency physician-hospitalist handoff interactions: development of the Handoff Communication Assessment[END_REF][START_REF] Siminoff | Cancer communication patterns and the influence of patient characteristics: Disparities in information-giving and affective behaviors[END_REF][START_REF] Street | The clinical context and patient participation in post-diagnostic consultations[END_REF]. Most studies have involved manually coding and classifying particular utterances or events in a conversation. There are a number of software tools, such as Transana1 and Noldus's products2 , that assist observers to mark up these events manually on a transcript or video, and then analyse them, for example through connections to statistics packages. A recent meta-analysis of the analysis techniques that are commonly used [START_REF] Connor | The analysis of verbal interaction sequences in dyadic clinical communication: A review of methods[END_REF] found that most studies, for practical reasons, focused on very short-term effects within a conversation -such as the influence of one participant's utterance on what the other participant says next. It also recommended that multiple analysis methods should be used, to better cope with the complexity of clinical communication. Besides the practical limitations of utterance coding studies, it is also clear that manual coding of a conversation is very laborious to perform Communications researchers are not the only academics who have an interest in analysing conversations. Within computer science, Artificial Intelligence and Natural Language Processing groups have developed a number of algorithmic techniques for analysing a conversation. For example, Leximancer [START_REF] Smith | Automatic extraction of semantic networks from text using Leximancer[END_REF], Latent Semantic Analysis [START_REF] Deerwester | Indexing by latent semantic analysis[END_REF], and Latent Dirichlet Analysis [START_REF] Blei | Latent Dirichlet allocation[END_REF] all analyse how words cooccur within a text to discover the flow of topics. It can be difficult to prove objectively that word co-occurrence is the same thing as a topic, but comparisons against human analysis have found that the algorithms' results correlate well with human judgement [START_REF] Smith | Evaluation of unsupervised semantic mapping of natural language with Leximancer concept mapping[END_REF][START_REF] Landauer | How well can passage meaning be derived without using word order? A comparison of Latent Semantic Analysis and humans[END_REF]. We are also aware of communications researchers who use these tools in their research, so the algorithmic approaches are gaining acceptance.

Applying an algorithmic analysis would make it feasible to analyse doctors' clinical conversations as part of their training or in professional development. However, we can go further. Just as Connor, et al, found in their review of manual analysis that it would be better to use multiple techniques to capture the full complexity of clinical communication, we believe the same is also true for automatic analysis. By combining different analyses, we can uncover more about the conversation. For example, colleagues of ours have developed algorithms for estimating the cognitive load of a speaker by analysing the sound of his or her speech [START_REF] Yin | Towards automatic cognitive load measurement from speech analysis[END_REF]. By combining that with the text analysis, we would be able to see both what participants are saying and how much thought they are having to put into their words. Each additional algorithmic tool can add an extra dimension to the analysis.

Concept of Operation

The analysis system we are developing has a visualisation client and a data server.

Both the client and the server accept plug-ins to extend their functionality. This extensibility is necessary as it is not certain which kinds of computer analysis will prove to be informative about a medical conversation, and which will not.

The server has to make as few assumptions as possible about the data it will be handling. Different analysis components use different kinds of data. For this reason, we often depict the data repository as a bucket, as shown in Figure 1. When a video is uploaded into the repository, the server begins to process it automatically. At each step in the processing, a plug-in takes some form of data out of the repository, processes it, and adds a new form of data back into the repository. One of the first steps in the workflow (one of the first plug-ins invoked) is likely to be to transcribe the video. This may be a semi-automatic step, in which a speech-totext engine produces a draft transcript, which is then edited manually using common transcription editing software. A second plug-in would take the transcript and process it through an NLP engine. A third plug-in would process the sound through a cognitive load analysis tool. Another plug-in might correlate the cognitive load data with the topic data for the video, and identify common stress patterns. The final result of the server processing is that there is a large amount of analysis data in the repository. The data can be played back as visualisations alongside the video, as depicted in Figure 2. Just as the server supports a plug-in architecture so that we can continually extend it with new kinds of analysis, the client also supports plug-ins so we can extend it with new kinds of visualisation. The mapping of server plug-ins to client plug-ins is not one-to-one. As we describe later in the paper, the presentation of the data has to be varied for a number of different kinds of user, so we expect there to be more visualisation plug-ins than analysis plug-ins. It is also worth mentioning that advice is a kind of visualisation. By providing an appropriate plugin, we can turn the visualisation plug-in into an explicit teaching tool.

Patient History Taking

One of the ways trainee doctors are taught patient history taking and communication skills is through simulated patients. An actor is trained to portray a patient scenario. The actor is trained to portray not only the history and symptoms of the patient, but also the emotional and personality characteristics. These portrayals are often standardised ("Standardised Patients"). The trainee doctor is asked to examine this simulated patient. At some institutions, these simulated patient sessions are video recorded for later review, rather than being reviewed live. Reviewing a video session thoroughly is a time-intensive task, so there are practical limitations on how many scenarios a trainee doctor can undertake.

As the sessions are already being recorded, they make an ideal first candidate for automatic analysis. The video recording can be processed by a transcriber, sound extractor, and multiple different analysis engines. This data can then be compiled, and the video played back together with the analysis results. Because the scenarios are standardised and well-understood, they make a very good research test-bed for analysis technologies. The fact that the educators already know what is wrong with the patient and what kinds of interactions to look for means that the analysis can be pre-seeded with information. This can help to mitigate any limitations of the analysis algorithms. If the algorithmic analysis proves successful, then we can begin to apply it educationally. Because algorithmic analysis would not involve the reviewer (though it would require time from the actors and someone to correct transcription errors) it would be possible for medical students to perform additional practice sessions using the system.

There have been previous attempts at automated teaching of patient history taking. Particularly, we are aware of interactive computer simulations of patients [START_REF] Bergin | Interactive simulated patient-an advanced tool for studentactivated learning in medicine and healthcare[END_REF][START_REF] Stevens | The use of virtual patients to teach medical students history taking and communication skills[END_REF]. The virtual patient listens to the doctor through voice recognition or free text entry, and responds through pre-recorded video. The computer simulations are necessarily lower fidelity than using a live actor, and they are also lower fidelity in their analysis than the multi-modal detailed algorithmic analysis we have proposed. We would be interested in augmenting a computerised simulation with fine-grained AI and NLP based communication analysis, to see whether it can be effective in that situation. As the computerised patient cannot vary its intonation, body language, and emotion as effectively as the human actor, however, it is possible that doctor-computer conversations would always appear somewhat stilted.

More Complex Use Cases

A second scenario we are interested in investigating is the shift hand-over meeting. Shift-handover is recognised as being a critical communication system, which has informational, social, educational functions [START_REF] Kerr | A qualitative study of shift handover practice and function from a sociotechnical perspective[END_REF]. Despite the hand-over's importance, though, there are comparatively few studies into factors that make a hand-over successful or unsuccessful. Likewise, in medical practice, it is difficult to monitor how well hand-overs are being performed.

We would like to see what automated AI and NLP analysis can reveal about these meetings. By recording hand-over meetings and processing them through the analysis tools, we hope to be able to inspect them in a way that is both more detailed and more efficient than manually coding utterances.

Technically, shift hand-overs are a very much more challenging problem than simulated patient scenarios. There are many more people involved in the meeting, and the number of communication channels (the number of possible two-way conversations) grows with the square of the number of participants. Similarly, whereas the simulated patient scenarios discuss a single medical case, the shift handovers discuss an entire ward. Furthermore, the shift cases are not standardised, so the software would not know ahead of time what the patients' conditions and issues are. Nonetheless, it may be possible to identify common failure modes in the communication. At the very least, we hope to uncover more detail about how the meetings function.

If a hand-over meeting is transcribed and analysed by a NLP and AI components, then this also provides a way of indexing the meeting, not just against words but against topics and other measures. If other clinical conversations are also recorded and indexed, it may be possible to forensically analyze how information is conveyed from one conversation to another. For example, it might eventually be possible to follow a piece of information from one medic's interaction with a patient, through a shift hand-over, to another medic's interaction with the same patient.

Status and Plans

We have constructed prototype software for our diagnostic tool, which is being continuously refined. We have begun to build our first processing and visualisation plug-ins, and have established collaborations with AI and NLP teams in order to add more. There is still, however, much work to be done before we can place an educational tool in front of a doctor. Firstly, we need to experiment with the tool. We need to process different kinds of conversation through various plug-ins, to find out what kinds of analyses combine well and add meaning -which combinations are more than the sum of their parts, and which are not. This is an experimental discovery process. Then, we need to work out ways of showing this data meaningfully to doctors. The data has been derived through complex algorithms, but must be understood by someone who has no knowledge of those algorithms, but who nonetheless works in an evidence-driven field and who will probably want to know the reasons not just the result.

The visualisations that the team needs during the discovery process might be different to those the doctors need in their professional development. So, we might find we need to develop visualisations and measures twice. The system is designed to be very adaptable, however, and can be configured differently for researchers than for doctors. We also hope to make it available to other researchers to use in their work.

Our first experiments are scheduled to take place in 2010, with the simulated patient scenarios. We also intend to apply the tool to a separate study comparing think-alouds by different practitioners. We hope that applying it in this study will help us to see how we can make the tool more useful for researchers. In 2011, we are scheduled to begin our investigations into hand-over meetings, and also hope to have developed a useful educational tool with the system.

Conclusion

The diagnostic toolbox we are creating is intended to be useful to both researchers and practitioners. We have a collaborative relationship with the Skills Development Centre of Queensland Health (the state hospital system), which gives us both the access to experimental subjects and scenarios that we need, and also a route to real impact if our research is successful. Our first goal is to uncover combinations of analysis that can provide meaningful insights into medical communication. Our second goal is to provide a useful educational tool that can present that meaningful analysis of communication to doctors, so they can understand more about their own communication. Through those two goals, we hope to provide a way of improving medical communication, at least in the area of patient history taking, and reduce the rates of patient harm.

Fig. 1 .

 1 Fig.1. The server repository acts as a generic bucket for data. When a video is uploaded, it begins to be processed through the various installed plug-ins automatically. In the figure, processing starts at the top-left and proceeds clockwise. Each plug-in takes data from the repository, passes it to an external processing component, and puts the resulting output into the repository as data that other plug-ins can then use in their processing.

Fig. 2 .

 2 Fig. 2. Stylised and real images of the client. The client plays back visualisations of the server data alongside the video.The visualisations are linked to the video, and so can also be used to navigate the video. The client supports a plug-in architecture, both so that it can be extended for new kinds of data, and also so that different visualisations can be used for different kinds of user.

http://www.transana.org

http://www.noldus.com

Acknowledgments

. The authors gratefully acknowledge their colleagues on the project: Harold Cruz and Michael Pickard. NICTA is funded by the Australian Government as represented by the Department of Broadband, Communications and the Digital Economy and the Australian Research Council through the ICT Centre of Excellence program.


Arnaud Tonnelier

Threshold Curve for the Excitability of Bidimensional Spiking Neurons

We shed light on the threshold for spike initiation in two-dimensional neuron models. A threshold criterion that depends on both the membrane voltage and the recovery variable is proposed. This approach provides a simple and unified framework that accounts for numerous voltage threshold properties including adaptation, variability and time-dependent dynamics. In addition, neural features such as accommodation, inhibition-induced spike, and post-inhibitory (-excitatory) facilitation are the direct consequences of the existence of a threshold curve. Implications for neural modeling are also discussed.

I. INTRODUCTION

The dynamics of spiking neurons is characterized by two distinct regimes monitored by the membrane potential, v, of the neuron: at moderate v values, typically lower than -55mV , the neuron acts as a linear filter, while for greater values of the membrane voltage (> -40mV ) the dynamics is highly nonlinear and follows a stereotyped trajectory, almost independent of the input, that represents an action potential or spike. At intermediate values, the neuron may switch in both regimes, depending on its internal state and the input. This feature defines the so-called neural excitability [1,[START_REF] Rinzel | Analysis of neural excitability and oscillations[END_REF] and relies on the existence of a threshold effect that is explicitly described in simple spiking neural models of integrate-and-fire type by a strict voltage threshold of the form v = θ where θ is a given voltage value, i.e., the intermediate regime, -55mV < v < -40mV , is reduced to a threshold line separating a subthreshold region from a super-threshold region. In contrast with standard integrate-and-fire neurons, detailed biophysical models of the Hodgkin-Huxley type do not have an explicit firing threshold and due to the continuous nature of the underlying differential system, there is a continuum of trajectories, from sharp action potential to small depolarization, making the concept of threshold ambiguous. However, it has been shown that the switching to the super-threshold regime occurs in a narrow range of input values, and a quasi-threshold [START_REF] Fitzhugh | [END_REF] effect is observed. This effect is commonly approximated by a strict voltage threshold [4,[START_REF] Gerstner | Spiking Neuron Models: Single Neurons, Populations, Plasticity[END_REF]. The partition of the phase space between a subthreshold region and a superthreshold region is closely related to the separation of time-scales between a fast action potential, with a time scale of order 1ms, and a slower subthreshold dynamics of input integration, of order greater than 10ms. When the fast dynamics is assumed to be instantaneous, the spike description is ignored, and the super-threshold dynamics is reduced to an instantaneous reset, making the concept of threshold central in neural modeling as a synthesis of all nonlinear effects. * arnaud.tonnelier@inria.fr Several drawbacks are associated with using a clear-cut voltage threshold. Properties such as threshold variability or threshold dependence upon the spiking activity can not be adequately described. To overcome these limitations, time-dependent voltage thresholds have been introduced as an ad-hoc description of threshold fluctuations but lack rigorous derivation [START_REF] Jolivet | [END_REF]. Recently, a voltage threshold equation has been derived from a quasi-static approximation of a Hodgkin-Huxley type model [7], and a series of work has revisited the concept of threshold manifolds in neural models [8][9][10]. Other threshold descriptions, including stochastic threshold models, have been introduced recently [11][12][13][14][15]. In this paper, we explore the threshold effect in bidimensional neural models and show that the strict voltage threshold v = θ [16] provides a poor approximation of the neural excitability whereas a threshold curve of the form g(v) = u, where u is a recovery (or adaptation) variable and g a non-monotonic function, accurately captures voltage threshold properties. Consequences for neural dynamics and neural modeling are discussed.

II. A THRESHOLD CURVE FOR BIDIMENSIONAL NEURAL MODELS

Sharp threshold models of neural firing offer an appealing framework to describe the neural activity and to analyze the neural dynamics. However, it has been long observed that the firing threshold is not a fixed number but rather a manifold [START_REF] Fitzhugh | [END_REF][START_REF] Izhikevich | Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting[END_REF]. Two well-known situations clearly illustrate the existence of a threshold manifold that partitions the phase space into two regions. (i) In type I neural models, where the stable resting state coexists with a saddle node, the stable manifold of the saddle point, called the threshold separatrix, is identified as the locus of threshold phenomena [START_REF] Fitzhugh | [END_REF][START_REF] Izhikevich | Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting[END_REF][START_REF] Fitzhugh | [END_REF]. (ii) In singularly perturbed systems, the firing threshold can be fairly well approximated by the middle branch of the Nshaped v-nullcline [START_REF] Izhikevich | Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting[END_REF]. In addition, a close link has been recently established between the neural excitability and the existence of canard trajectories [8,9], emphasizing the crucial role of the middle branch of the v-nullcline together with the existence of multiple time scales.

Here, we explore the concept of threshold in some popular neural models. For clarity and simplicity, we mainly consider bidimensional (2D) neural models where the time-dependent activity of the neuron is described by the membrane potential, v(t), and an additional variable, u(t), that accounts for adaptation, recovery processes, or, more generally, voltage-gated channels dynamics (sodium channel inactivation or potassium activation). Most often, the variable u acts as a negative feedback on v. A well established stimulation paradigm for excitabilitythreshold extraction uses an instantaneous voltage change, i.e., a pulse of zero duration is applied to the membrane potential equation. For membrane potential above a critical voltage value, a spike is elicited and the recorded value defines the fast-depolarization threshold that may be obtained in-vitro using a short current pulse. This differs from the slow-depolarization threshold even though a close link exists between both thresholds [7]. In both cases, the recorded voltage threshold depends on the state of the neuron at the time of the stimulation, and we generalize the threshold-extraction method using an instantaneous shock of all variables. Taking a point in the phase space as an initial condition, we determine if a spike is elicited or not. More precisely, we compute the peak value of the membrane voltage, and a graylevel is assigned to the result: black regions indicate a spike and white stands for a small subthreshold response. Intermediate gray levels are used for intermediate responses. An arbitrary long time horizon is used to detect the peak value of the membrane voltage but, in practice, a maximum is reached in finite time, corresponding to the so-called spike-latency. The response surface of a linear system would be a grayscale gradient, i.e., with a clear distinction of all the intermediate responses, whereas an idealized on-off neuron model would lead to a binary black-and-white picture. The response surfaces of the bidimensional Abbott-Kepler reduction [19,20], the Krinsky-Kokoz reduction [1,21], the FitzHugh-Nagumo model [22,[START_REF] Nagumo | Proc. IRE[END_REF], the Morris-Lecar model [START_REF] Morris | [END_REF], the adaptive quadratic integrate-and-fire (QIF) model (Izhikevich model [25]) and the adaptive exponential integrate-andfire (EIF) model [26,27] are plotted in Fig. 1. Some characteristic elements are also displayed: the nullclines and the stable manifold of the saddle fixed point, when it exists. The plots clearly show the existence, in the phase plane, of a tiny region where the peak value of the membrane potential blows up, delimiting a subthreshold region from a super-threshold region. This boundary defines a curve which is poorly approximated by a vertical straight line, v = θ, but rather by a threshold equation of the form

u = g(v) (1)
where one piece of the g curve is well fitted by the middle branch of the v nullcline. At hyperpolarized v values, the threshold curve is flat (Fig. 1C,E,F) or shows a hump (Fig. 1A,B). For a type I model (Fig. 1E, F), i.e., presenting a saddle point, the stable manifold of the sad- dle point achieves a good approximation of the threshold manifold, corroborating previous results on the characterization of neural excitability. A reasonable choice for the threshold curve (1) is the sigmoidal curve defined by the Boltzmann function

g(v) = a 1 + e -λ(v-θ) + b (2)
where b acts, at low v values, as a threshold on the recovery variable, θ determines the locus of the transition of the sigmoidal function, and λ gives the stiffness of the transition. A strict voltage threshold is retrieved in the limit λ → ∞. To capture the fold that may occur near the resting state, a combination of two sigmoidal curves may be used. In this case, the function g in ( 1) is not invertible, and it is not possible to derive a threshold equation of the form v = h(u). To assess the validity of the concept of threshold curve and to estimate the quality of its approximation by Boltzmann functions (2), an enlargement of the voltage response in a relevant area is displayed in Fig. 2 for the Abbott-Kepler reduction and the FitzHugh-Nagumo model.

To investigate the dependence of the threshold curve on the applied constant input current (DC current), we extract the threshold curve of the bidimensional Abbott-Kepler model (Fig. 3A) and of the FitzHugh-Nagumo model (Fig. 3B) for different input values. As can be seen, the injected DC current does not affect significantly the threshold curve except at hyperpolarized voltage values where a vertical shift of the threshold curve, i.e., along the u-axis, is observed. In addition, the sensitivity to inhibitory input can be enhanced due to a more pronounced hump as the DC current increased (Fig. 3A). Two-dimensional reductions of detailed neural models rely on the separation of time scales where it is commonly assumed that the fast dynamics, typically the sodium activation, is instantaneous. However, the sodium activation dynamics has an impact on the voltage trajectory of the neuron, and the main discrepancy between the original model and its reduction arises from the assumption that the fast time scale is instantaneous; i.e., the activation of sodium channels follows instantaneously the variation of the membrane potential [28]. To assess the impact of fast dynamics on the threshold curve, we consider the three-dimensional Abbott-Kepler reduction obtained by lumping together the slow variables of the Hodgkin-Huxley model and retaining the exact sodium activation dynamics [28]. We compute the threshold manifold in the three-dimensional state space, and we display in Fig. 4 five different projections (curves) on the (v, u) plane. We see that, as the sodium activation increased, the threshold curve is progressively shift to the left, i.e., the sensitivity to excitatory input is enhanced. This effect may be accounted for using θ ← θ + µm in [START_REF] Rinzel | Analysis of neural excitability and oscillations[END_REF] where m is the sodium activation variable and µ > 0 measures the sensitivity of the threshold surface to the sodium channel dynamics.

III. GENERALIZATION AND CONSEQUENCES

Let x = (v, u 1 , . . . , u n ) be the vector describing the neural activity where v is the membrane potential and 0 < u i < 1 are the activation or inactivation variables of ionic channels. Spikes are generated whenever a threshold condition is fulfilled that can be written in a general and abstract setting as G(x)(t) = 0 where G is a functional taking the trajectory x(s), s ≤ t as an input; the excitability threshold at time t depends on the past history of the neuron state variables. Integrate-and-fire models assume the simplest form of threshold function where G(x) = vθ; the firing of a spike is determined comparing the membrane potential at the present time to a given fixed value. Based on the observations in the previous section, we define the following threshold equation:

g(v(t)) = n i=1 α i u i (t) (3)
where the weights α i are constants whose sign determines its influence on the threshold: a negative weight facilitates excitation whereas a positive weight decreases the neural excitability, i.e., the effective voltage threshold is shifted to the right (at depolarized voltage values).

In the following, we discuss the consequences of the threshold equation ( 3) on both neural dynamics and neural modeling.

A. Adapting Threshold

Let us consider the threshold hyperplane given by

v(t) = θ 0 + n i=1 α i u i (t) (4)
obtained from (3) using the threshold function g(v) = vθ 0 , which can be seen as a natural generalization of the strict voltage criterion used in standard integrateand-fire models. Let us assume that each voltage-gated variable u i follows the v-dependent first-order kinetics

τ i (v) ui = u ∞,i (v) -u i , for i = 1, . . . , n
, where in the subthreshold regime both functions τ i (v) and u ∞,i (v) are assumed to be nearly constant and, in the super-threshold regime, the dynamics of u i is reduced to an instantaneous change u i ← u i + d i , i.e., ion channels rapidly open (or close) at high voltage values (or equivalently τ i (v) ≪ 1 during a spike). Such spike-induced currents are traditionally used to describe adaptation or refractoriness. Let t k be the successive firing times of the neuron. The relaxation dynamics of u i between firing events combined with the jump condition at firing times lead to

u i (t) = u ∞,i + k h i (t -t k ), (5)
where

h i (t) = d i exp(-t/τ i )H(t),
with H is the Heaviside-step function. Using (5), the threshold hyperplane equation ( 4) can be rewritten as a dynamic voltage threshold v(t) = θ(t) where θ(t) is the time-dependent threshold given by

θ(t) = k h(t -t k ) + θ0 , (6)
where

h(t) = n i=1 α i h i (t), θ0 = θ 0 + n i=1 α i u ∞,i .
We recover here the multi-timescale adaptive threshold (MAT) model introduced in [29] as a generalization of the standard adapting threshold model. The MAT model is therefore a specific instance of threshold hyperplane models where the number of threshold time constants is given by the number of voltage-gated variables playing a role in the threshold hyperplane, the different time scales of the MAT model are given by the time constants of the gating variables, and the weights associated with the time constants are given by the size of the jump of the related activation (or inactivation) variables (when the neuron assumes a spike) multiplied by their weights in the threshold hyperplane equation.

From [29], it is expected that a threshold hyperplane with two ionic currents, and thus having two distinct time constants, is sufficient to achieve very good predictive performances. For identical ionic time constants, i.e., ∀i τ i = τ , the threshold hyperplane is equivalent to the dynamic threshold

τ dθ dt = θ0 -θ, (7)
with the instantaneous jump θ ← θ + i α i d i each time a spike is generated.

B. Dependence of the Voltage Threshold Upon the Subthreshold Activity

The threshold curve, Eq. ( 3), induces a dependence of the voltage threshold upon the subthreshold activity. This is illustrated in Fig. 5 where the 2D Abbott-Kepler reduction is stimulated by the subthreshold current

I syn = g exc (v -E exc ) + g inh (v -E inh )
mimicking the stochastic arrival of excitatory and inhibitory presynaptic action potentials, modeled by homogeneous Poisson processes. Parameter g exc (g inh ) is the excitatory (inhibitory) conductance consisting of the summed input from excitatory (inhibitory) synapses. Parameters E exc and E inh are the corresponding reversal potentials. The voltage trace is compared with the voltage threshold extracted from the threshold curve of the Abbott-Kepler model (see Fig 1A). The U shape of the threshold curve implies the existence of two voltage thresholds: an excitatory threshold that is reached for sufficiently strong excitation and an "inhibitory" threshold that is reached for hyperpolarizing current. Both thresholds show fluctuations that are correlated to the subthreshold voltage trajectory. When inhibition is blocked (at time t = 100ms, Fig. 5) a spike is emitted as predicted by the crossing of the excitatory threshold. After the spike, there is a short period where both thresholds are not defined producing a natural refractory period. Most of voltage-dependent threshold models are, in fact, spike-dependent where the threshold trajectory is updated whenever a spike is emitted and subsequently decays toward a resting value (as illustrated in sec. III A). However, it has been observed for quite some time that the threshold also depends on the subthreshold activity, and, remarkably, well before the precise knowledge of neuronal ionic channels, Hill [30] proposed a dynamic threshold that rises with the subthreshold voltage. A threshold that depends on the voltage trajectory, and not only on the spike events, produces a rich phenomenology [31] and has been recently used to model post-inhibitory facilitation (PIF) [32,33] and type III excitability [34].

We show here (see Appendix A) that such dynamical threshold models can be derived from bidimensional integrate-and-fire models with a threshold line where the dynamics of the recovery variable determines the properties of the voltage threshold.

C. Dependence of the Voltage Threshold Upon the Input

An implicit dependence of the voltage threshold upon the input is generated by the threshold criterion (3). We illustrate here this property using the spike-responsemodel

v(t) = η(t -t f ) + ∞ 0 κ(s)I(t -s)ds (8)
where η describes the shape of the spike and κ the response to an incoming short pulse. The spike timing t f is the time at which the membrane potential crosses the threshold that we define here as the threshold line

v(t) = θ 0 + αu(t) (9)
where u is an auxiliary variable given by u = a(bvu). Parameter a > 0 determines the time scale of u, and bv describes the influence of the voltage subthreshold fluctuations on u. Integrating the u-dynamics, the threshold condition can be rewritten as

v(t) -αab t 0 v(s)e a(s-t) ds = θ 0 , (10)
that depends on the voltage trajectory between spikes. A non-zero initial value of u will translate into a shift for θ 0 in (10): if at time t = 0 variable u(t) is reset to u r , the effective threshold becomes θ0 = θ 0 + αu r . In Fig. 6 a slow ramp input is injected to the SRM model (8) with the threshold condition (10). As can be seen, the measured voltage for spike initiation is greater than the one obtained from a short current pulse (horizontal straight line). Due to the strict voltage criterion commonly used in SRM-type models, the threshold difference between a slow ramp current and a short current pulse can not be reproduced. This limitation is overcome using the threshold line (9) where parameter α > 0 mimics the influence of K + activation or N a + inactivation on the spike threshold. A negative α value accounts for spike facilitation.

D. Voltage Threshold Variability

We simulate the 2D Abbott-Kepler model with a super-threshold current and indicate in Fig. 7 the voltage values at which the threshold curve is crossed. In contrast with strict voltage threshold models, the threshold curve reproduces the small fluctuations of the critical voltage for spike initiation. The variations of the voltage threshold depend on the input current making the prediction of spike timings with fixed voltage threshold models uncertain and very sensitive to the firing value [4,[START_REF] Jolivet | [END_REF]. In fact, the predictive quality of pure voltage threshold models severely degrades for small amplitude fluctuations of the input current due to a poor approximation of the effective threshold. One expects that the use of a threshold curve overcomes the issue of threshold fitting and provides a simple framework to improve the accuracy of simple predictive models.

E. Model Reduction and After-Spike Resetting

To avoid the simulation of the peak and the downstroke of an action potential, threshold rules are desirable to stop the simulation when a spike is initiated and to restart the integration after the spike duration. Therefore, in addition to the threshold criterion, one has to determine a restarting time together with initial values for state variables. Due to the low variability of spike duration, a fixed refractory period is commonly used. This period is followed by an update of the state variables that may be a "hard" after-spike reset x ← x r or an additive process x ← x + d. The former is mainly used to describe the after-spike update of the membrane potential, whereas the latter is commonly used for the recovery variable to account for adaptation. The resulting models belong to the class of integrate-and-fire models. We propose here an integrate-and-fire approximation of the 2D Abbott-Kepler model where the threshold curve is defined as a combination of two sigmoidal functions. The duration of a spike is assessed using different initial conditions on the threshold curve, and the update rule is fitted by a hard reset for both variables. Parameter values of the reduced IF model are given in Appendix B. To assess the quality of the approximated IF model, we compare the trajectory of the original model with the one of the reduced model, both driven by the same fluctuat- ing current. As shown in Fig. 8, the spike timings are correctly predicted by the threshold curve. In addition the subthreshold voltage trajectory is reproduced with a very high precision (trajectories are almost indistinguishable) indicating that both a constant spike duration and a hard reset for the two variables accurately capture the behavior of the original model. Note that the simulation restarts immediately after the sharp downstroke of the action potential, i.e., the spike afterpotential is simulated by the IF approximation.

F. Piecewise Approximation of Neural Models

A simple and numerically efficient approach to capture the threshold curves of spiking neurons is to use an idealized piecewise linear U curve made of two distinct parts, located in the hyperpolarized and depolarized region, called, respectively, the left branch and right branch hereafter. More precisely, the threshold curve is split into two pieces, defined by x.n e = 0 and x.n i = 0, that approximate the right and left branches of the threshold curve, respectively (see Fig. 9), where x is the state vector x = (v, u) (for convenience, the origin is shifted to the turning point of the threshold curve), n e = (-sin θ e , cos θ e ), and n i = (sin θ i , cos θ i ). Angles θ e and θ i monitor the threshold location: θ i is related to the hyperpolarized spike threshold (or "inhibitory" threshold), and θ e measures the degree of excitability with respect to excitation. For θ i = 0, the left part of threshold curve is flat, and inhibitory inputs can not elicit spikes.

When θ e = π/2, we recover a pure voltage threshold materialized by a vertical straight line in the phase plane. Depending on the branch of the threshold curve that is reached during the activity, different spike shapes (including different spike latencies) and different after-spike rules may be used. In particular, the spike latency observed in the 2D Abbott-Kepler model after a brief inhibitory input (simulations not shown) can be accounted for in its integrate-and-fire reduction adding a given precalculated latency.

Similarly to the piecewise-linear threshold, the nonlinear spike-generating currents of bidimensional spiking models may also be approximated by piecewise-linear functions defining a new class of IF models that are computationally cheap, amenable to event-driven simulations and analytically tractable. In addition, they are endowed with properties associated with fluctuating voltage thresholds.

When the dynamics of the membrane potential is mainly driven by the membrane voltage, i.e., u provides a negligible feedback on v, the auxiliary variable plays a role only in the definition of the threshold curve and the model can be further reduced to a one dimensional differential equation where the influence of the auxiliary variable is directly incorporated in the threshold criterion (see Eq. 10) where the convolution with the exponential kernel is responsible for a dependence of the threshold on the past to recent activity of the neuron.

G. Post-inhibitory Spikes, Accomodation and Facilitation

The two distinct branches of the U-shaped threshold curve are responsible for different dynamical features. The left part of the threshold curve, mainly located at hyperpolarized voltage values, induces postinhibitory spikes: a strong enough brief inhibition pushes the trajectory leftward and drives the phase point across the hyperpolarized threshold evoking a rebound spike. Hyperpolarization-induced activity has been already reported in different models. Voltage-dependent dynamic threshold can exhibit spikes induced by fast inhibition [31,33]. A modified type I FitzHugh-Nagumo model may fire at hyperpolarized states [35]. These post-inhibitory spikes are closely related to the fold of the threshold curve near the rest state resulting in a second excitation threshold at hyperpolarized membrane potential. It is worth noting that this property differs from the well known phenomenon of anodal break excitation (or post inhibitory rebound) where a neuron fires after being released from a prolonged inhibition [36,37]. The right branch of the threshold curve generates the so-called phenomenon of accomodation [30]: the critical voltage value for spike initiation obtained from slowly increasing input current is greater than the one derived from a very short current (see Sec. III C). The rise of the voltage threshold is monitored by the time constant of the recovery variable entering in the definition of threshold line and poorly depends on the membrane voltage equation. As v depolarizes, the auxiliary variable grows and the effective voltage threshold becomes larger. Therefore, spikes occur preferentially when the negative feedback does not operate on the threshold, i.e., when the rise of the voltage is sufficiently fast compared to the auxiliary variable dynamics. Such a mechanism amplifies coincident inputs [START_REF] Azouz | Proc. Natl. Acad. Sci[END_REF]. An additionnal consequence of the excitatory threshold curve is the PIF phenomenon [32]: a subthreshold inhibitory event occuring in a favorable time window can facilitate spiking in response to a subsequent subthreshold excitation. The slope of the right branch of the threshold curve determines the degree of facilitation whereas the window of opportunity is controlled by the time constants of the model. The hyperexcitability period occuring in a time window preceding a subthreshold inhibitory input may result in a successful IE (inhibitoryexcitatory) pair, and, by a symmetric and similar mechanism, a successful EI pair may be obtained. In fact, due to the U-shape of the threshold curve, post-excitatory facilitation (PEF) may occur in symmetry to the PIF effect: a subthreshold inhibition may elicit a spike if a preceding excitation is applied in a favorable time-window. However, the PEF phenomenon is less pronounced due to the attenuation of the left part of the threshold curve compared to its right part. Interestingly these different features may be obtained independently of the nature of the rest state (node or focus, i.e., resonant or not [37]). In particular, it is not necessary to have an overshot of the membrane voltage to observe post-inhibitory spikes or PIF. It is therefore possible to design an integrate-and-fire model of purely integrator type that exhibits post-inhibitory facilitation and may therefore prefer a certain resonant frequency. However, such a resonant behavior may be enhanced when the neuron acts, near the rest state, as a damped oscillator [37]. The dynamical properties generated by the two branches of the threshold curve may interplay. A resonant response associated with the left branch of the threshold curve (responsible for postinhibitory spikes) may interfere with resonant properties related to the right branch (responsible for postexcitatory spikes) leading to a com-plex sensitivity to inhibitory-excitatory inputs. Moreover, it is well known that a brief inhibition can suppress the neural response to a super-threshold excitatory input. Conversely, a brief excitation can also cancel a post-inhibitory spike and excitation can therefore reduce the firing rate induced by inhibition [33]. Finally, PIF and PEF can occur simultaneously, resulting in output spikes evoked by combinations of subthreshold inhibitory and/or subthreshold excitatory events (that would be below threshold when delivered alone). This challenges the classical view of the role of inhibition in shaping the processing of spike trains [START_REF] Luk | [END_REF].

IV. DISCUSSION

Due to its stereotyped nature, almost independent of the input, and its weak role in neural processing, the super-threshold regime of neural models is commonly reduced to a formal event, the so-called spike event. In this view, hybrid dynamical systems (HDSs) naturally arise [40,41] to model the smooth excitable behavior of subthreshold integration combined with an after-spike update of the state variables (referred to as the jump transition map in the HDS terminology) when a threshold criterion (also defined as the autonomous jump set) is fulfilled. The subthreshold properties of neural HDSs have been largely discussed, and we place here our attention on the threshold equation, the so-called switching manifold in control sciences.

In this paper, we analyze the threshold manifold of bidimensional spiking neural models and show that the commonly used voltage equation v = θ poorly approximates the switching manifold of neural models. We suggest here a threshold criterion of the form u = g(v) (which is preferable to v = h(u); i.e., g may be non invertible) and show that such a threshold curve accounts for numerous features of spiking neurons traditionally associated with dynamical voltage threshold models. However, in contrast to conventional dynamical threshold models, the voltage threshold, defined by the threshold curve, may depend on subthreshold voltage fluctuations, and not only on spiking activity. Depending on the dynamics of the auxiliary variable entering in the definition of the threshold criterion, different properties can be observed. For subthreshold-activated ionic channels (low-threshold currents), the associated threshold curve leads to a dependence of the voltage threshold upon the subthreshold activity and plays a significant role in post-inhibitory facilitation, type III excitability, and adaptation. Dynamical voltage thresholds that depend on the spiking activity can be derived from a threshold curve model where the auxiliary variable accounts for spike-triggered currents. In this framework, the time constants and the stationary value of the voltage threshold are directly related to the associated ionic channels. Different modeling perspectives are offered by the con-cept of threshold manifold. For instance, a simple onevariable leaky integrate-and-fire model can show hyperexcitability provided that the threshold equation incorporates a voltage-gated feedback. Features that are intuitively associated with resonate-and-fire models can also exist in integrator type models, as post-inhibitory spikes, as post-inhibitory facilitation, or as sensitivity to doublets. A non-monotone threshold curve implies the existence of two voltage thresholds at both hyperpolarized and depolarized voltage values, challenging the common view of the role of inhibition in reducing excitability by pushing the system far from its spike threshold. This gives a more subtle concept of nearness to threshold than the one traditionally believed. can be recast as the nonlinear integrate-and-fire model (A1) where the equilibrium function (A2) is now approximated by u ∞ (v) ≈ u a e λ (v-va) which is obtained from (A2) for v a -values larger than subthreshold voltage values, i.e., in the limit e λ(v-va) ≪ 1. In type III excitable neuron, the time scale of the voltage threshold (which is the time scale of the negative feedback, u(t)) is comparable to the neuronal time constant enabling a competition between the voltage rise and the negative feedback, and precluding spiking if the rise of the membrane voltage is too slow. The threshold curve, u = g(v), of the 2D Abbott-Kepler reduction is approximated by a linear combination of two Boltzmann functions where parameters are determined using a least square method. We find g(v) = -71 + 23.2 1 + e -0.18(v+54.4) + 6.53 1 + e -0. 24(v+72.2) .

To estimate the spike duration and to derive the update rules, we use several initial conditions on the threshold curve. Two cases occur depending on the threshold crossing location. When the left part of the threshold curve is crossed, the spike shape shows different latencies, i.e., a delayed spike initiation is observed. The other part shows a poor variability and a spike duration of around 2ms is observed. In all cases, the after spike update is well approximated by a hard reset for the membrane potential, v r = -76mV , and for the recovery variable, u r = -46mV .

FIG. 1 .

 1 FIG. 1. Voltage response of bidimensional neural models. The gray level of a pixel indicates the maximum value reached by the membrane potential when the initial state of the neuron is where that pixel is located in the phase plane (v, u) (white is for low-voltage values and black is for high-voltage values). The corresponding voltage response is shown for (A) the 2D Abbott-Kepler reduction, (B) the Krinsky-Kokoz reduction, (C) the FitzHugh-Nagumo model, (D) the Morris-Lecar model, (E) the adaptive QIF and (F) the adaptive EIF. The nullclines are plotted (solid lines) together with the stable manifold of the saddle point (dashed lines) for the two last models (almost indistinguishable from the black region boundary).

FIG. 2 .FIG. 3 .

 23 FIG. 2. Enlargement of the voltage response of (A) the 2D Abbott-Kepler reduction and (B) the FitzHugh-Nagumo model together with the corresponding threshold-curve approximation with (B) one or (A) two Boltzmann functions. The approximation closely matches the boundary between the spike region (in black) and the small voltage-response region (almost white). As in Fig. 1, the nullclines are shown.

FIG. 4 .

 4 FIG. 4. Influence of the fast-sodium activation on the threshold curve. Projections of the threshold manifold of the threedimensionnal Abbott-Kepler reduction in the (v, u) plane for different values of the sodium activation variable m, m = 0.01, 0.1, 0.2, 0.3 and 0.8 (from right to left).

FIG. 5 .

 5 FIG. 5. Voltage trajectory of the 2D Abbott-Kepler model subject to conductance injection. The neuron is driven by an excitatory-inhibitory balanced input that maintains the membrane voltage between the two fluctuating voltage thresholds shown in the dot-dashed line (upper) and in the dotted line (lower). At time t = 100ms the inhibition is blocked and a spike is elicited. The inset shows a zoom of the region where the excitatory threshold is reached.

FIG. 6 .

 6 FIG.6. Dependence of the threshold upon the input. Stimulation of the spike-response-model with a slow ramp input current. When the membrane potential reaches the threshold (dotted lines), a spike is emitted. The thin horizontal straight line indicates the spike threshold obtained from a brief current pulse.

FIG. 7 .

 7 FIG. 7. Simulation of the 2D Abbott-Kepler reduction with fluctuating conductances. Black dots indicate the voltage value at which the threshold curve is reached. As predicted, a spike is initiated when the threshold condition is fulfilled.

FIG. 8 .

 8 FIG. 8. Simulation of the 2D Abbott-Kepler model (dotted lines) and the corresponding integrate-and-fire approximation (solid line).

FIG. 9 .

 9 FIG.9. Piecewise linear threshold for bidimensional spiking neuron. When the spike region is reached, i.e., the threshold curve is crossed, the trajectory restarts from the reset curve (v = -80mV ).

  Appendix B: Integrate-and-fire Approximation of the 2D Abbott-Kepler Model.

Appendix A: Integrate-and-Fire Neurons with a Threshold Line to Model Post-inhibitory Facilitation and Type III Excitability.

Let us consider the 2D integrate-and-fire model

where a spike is emitted when the threshold line vu = θ 0 is crossed. Variable u provides a negative feedback on v whose strength is determined by ǫ > 0. Let us assume that the corresponding ionic current has little effect on the subthreshold dynamics of v(t), i.e., ǫ ≪ 1. Therefore u acts on the dynamics only through the threshold criterion. We further assume that (i) the relaxation time τ u is almost constant during the subthreshold regime and (ii) the equilibrium function, u ∞ (v), is given by the sigmoidal function

For small λ values the steady state function (A2) can be approximated by:

Let us define θ(t) = θ 0 + u(t), α = u a λ/4, θ0 = u a /2αv a + θ 0 , and τ θ = τ u . Model (A1) can be rewritten as the idealized PIF model introduced in [32]:

with the voltage threshold condition v(t) = θ(t).

Similarly, the nonlinear threshold dynamics τ θ dθ/dt = a + e b(v-c) -θ used to describe type III excitability [34]


David Calder
email: david.calder@cbs.curtin.edu.au

Multimedia Speech therapy Tools And Other Disability Solutions as Part of A Digital Ecosystem Framework

Keywords: Multimedia, computer-based therapy, speech therapy, cardiovascular accident, assistive technology, sound interface displays, laser, disabled, infrared, long cane, portable electronic device, sensory channels, visually impaired, ultrasonic pulse-echo, ambient sound cues

Curtin University has developed a multimedia prototype system for use by speech therapists to assist in the rehabilitation of motor speech impaired patients. These are often stroke victims who have to relearn the ability to communicate effectively. The process is usually laborious and instead of the usual card-based prompts used by the therapist, this multimedia solution offers screen and audio prompts together with high quality digitised speech. The result is a reduced work load for the therapist. In parallel with this work, is a Mobility Aid Assistive Device Program for the visually impaired. The support clusters for all these groups, whether they be therapists, caregivers, manufacturers of hardware or software do not interact to the extent they might. The author proposes a collaborative digital ecosystem framework that would assist this challenge.

Introduction

There is a worldwide shortage of speech training services to accommodate the clients who require speech therapy [START_REF]ASU Designs Program To Address Shortage of Speech Therapists, Phoenix Arizona News[END_REF]. Cardiovascular accidents are the third largest killer in Australia after heart disease and cancer. Those that survive this trauma usually have need for rehabilitation. Motor impairment of the speech organ is a common occurrence. We have produced a Multimedia system which replaces the chart and paper-based cues found in many speech therapy units. The aim is to assist speech therapists by decreasing the time they have to spend in direct contact with patients. More importantly, there are also a number of benefits to clients and these will be discussed later.

As part of the Digital Ecosystem and Business Intelligence Institute (DEBII) research team at Curtin University, we worked closely with speech professionals from Royal Perth Hospital in order to tailor the interface exactly to their requirements. We were guided by practicing speech therapists in ensuring existing and well established interactive therapy strategies for certain classes of dysarthria, were followed as closely as possible. The first version of the system has undergone beta testing in Western Australia. This is a straightforward software solution using standard personal computers already available to therapists within their environment.

The author has also been researching assistive devices for the visually impaired and portable speech aids for the speech impaired. These, on the other hand, require dedicated hardware and the design of extreme demand interfaces in order to offer the most appropriate solution. However, the support infrastructure clusters, for each of these above examples, do not collaborate or communicate to any great degree.

2

Cerebro-vascular accident

When the oxygen supply to the brain is blocked or when an artery in the brain is ruptured, a common stroke results. The former is the result of a clot traveling to the brain and occluding blood flow in a cerebral artery [START_REF] Hewer | Stroke -A Practical Guide Towards Recovery[END_REF]. Once the oxygen supply has ceased, hydrogen ions propagate within the brain cells and damage the cells. Within four to eight minutes, the cells begin to die, and of course this destruction is irreversible. In the case of a reduced blood flow, cells may stop functioning but later recover when normal perfusion is restored [START_REF] Kaplan | The Total Economic Costs of Stroke in Australia[END_REF]. Cardiovascular accidents are the third largest killer in Australia and in many other countries of the western world. The highest killer is heart disease followed by cancer [4]. Fifteen percent of people over the age of forty living in the western world die of cardiovascular accidents. A further fifteen percent are also stroke victims who survive and require some form of rehabilitation [START_REF] Hird | Communicative Competence, Self-care and Mobility in Left and Right Hemisphere Adult Stroke Patients[END_REF]. Of this group five percent make a full recovery whilst ten percent suffer some permanent disability [START_REF] Dyken | Risk Factors in Stroke: A Statement for Physicians by the subcommittee on Risk Factors and Strokes[END_REF].

As medical technologies have improved and the relative death rate has fallen, so the need to offer suitable rehabilitation services to the victims who have survived this trauma has increased. Patients are left with multiple disabilities and these often include loss of speech. The anterior portion of the brain is associated with speech production and damage to this area can result in motor speech disorders [START_REF] Darley | Motor Speech Disorders[END_REF]. The severity of the motor speech disorder may vary from person to person but even where spontaneous recovery is evident, some speech rehabilitation will probably be carried out. In the case of more severe speech impairment, long term rehabilitation may result [8]. In most instances, the road to recovery is long and difficult, placing stress on both client and therapist. Any means that may assist this load for both the client and the therapist should be investigated.

3

Hardcopy patient cue cards

During a standard speech therapy session, the client is presented with a number of cues which include symbols representing phonemes/vowels, drawings indicating air flow through the mouth and nose and the many combinations of lip, teeth and tongue positions which are essential in general articulate speech. Cards were identified as part of a major problem by speech professionals during the initial data analysis of the old system at Royal Perth Hospital, Shenton Park in Western Australia. The therapist is under pressure to find the correct flash card or phoneme chart as she sits with the client. Together with these paper-based cues she uses her own speech and face to offer sound and visual cues to the client. In order to produce the consonant M for example, the client may be asked to place the lips together and allow the air to flow through the nasal cavity. The therapist will give an example by producing the sound herself and simultaneously accentuate the positioning of the lips.

The client is then asked to repeat this step. Consonant and consonant/vowel combinations such as this need constant repetition by the client and consequently the therapist before any improvement is expected. This face-to-face contact can be very tiring for both the client and the therapist. As therapy is done on a one to one basis, duration of contact per individual per week may be limited to only a few hours. Clients are therefore being deprived of the continuous training which would benefit their rehabilitation.

4

Multimedia Therapy Tool

The concept of a system to assist in speech therapy resulted from earlier work with speech therapists in the United Kingdom in developing a speech aid prototype [8].

The development of the therapy tool depended on a requirement to address some of the most significant problems associated with conventional therapy methods. The Department of Speech Pathology at the Royal Perth (Rehabilitation) Hospital, Australia, was involved in the development of the system and provided the old model of operations for analysis from which the computer-based system was developed. Assistant Professor G. Mann, now in the Department of Communication Disorders at the University of Florida and widely recognised as a key authority, worked closely with us over several years in developing the original version of the multimedia tool [START_REF] Mann | The Mann Assessment of Swallowing Ability[END_REF].

The therapy tool aims to relieve constant therapist/patient supervision, particularly where time consuming repetitive tasks are involved. Therapists can use their time more effectively in planning new goals whilst the computer provides visual and sound cues to the client. Therapists no longer have to organise cue cards or sort through hundreds of icons and drawings. These were seldom in colour whereas the computerbased system augments these traditional methods by using colour and animation. The latter was something that could not be achieved on loose pieces of paper or cardboard! Consequently the therapy process can run more smoothly and effectively as all cues are presented on the screen and/or produced by the high quality stored speech system.

There are a number of other benefits which particularly relate to patients and have been highlighted during beta testing. It has been found that the system could be used at home as well as in the conventional environment of the speech therapy unit. Where a client is left with the computer and removed from the clinical surroundings, the stress of embarrassment is removed. This is particularly evident when an older client has to "relearn how to speak" in the presence and under the direction of a young speech therapist. The stress placed on the client during these encounters should be seriously considered.

Another bonus for the system is that it is based on a standard IBM compatible PC and could even be run from a portable notebook mounted to a wheelchair. Most households now have several computers with a more than adequate built-in sound systems [START_REF] Edwards | Design of Speech-Based Devices[END_REF]. This means that in most instances, only the software need be installed in the home to allow therapy to continue. Other members of the family could be involved in the rehabilitation and the clinical sessions at the hospital could be used for the monitoring of progress and not be associated with intrusion into the privacy of the patient [12]. Communication within the family is an important part of rehabilitation and the therapy tool could help in promoting this. The high cost of therapy is relieved but not replaced by this augmentative system [START_REF] Hodge | Why is the potential of augmentative and alternative communication not being realized? Exploring the experiences of people who use communication aids[END_REF].

Since the therapy tool saves clinician time, it may also help relieve the shortage of speech therapy services. If the amount of time spent with each client is reduced, the clinician could then take on a greater case load.

The nature of a computer-based system is that, "it doesn't have to see another patient in an hours' time". Therefore clients may train at their own pace under no stress. Sound and visual cues may be repeated over and over again without the pressure of supervision. Of course not all patients would be suitable candidates for this level of freedom but initial tests have indicated great success for certain motor impaired victims.

5

Multimedia Animation and Stored Speech

Asymetrix Multimedia Toolbook was used to develop this system. It operates in a high level, Windows-based object-oriented programming environment. Toolbook uses a book metaphor for multi screen production. Each screen is described as a page and all pages in a production are called a book. Making use of these properties pages were constructed to emulate the icons and visual cues used in conventional therapy. Animated visual representations of the vocal system were added to the standard set of cues. High quality digitised speech was recorded from a practicing speech therapist so that an exact representation of a set of speech sounds could be achieved. As the normal therapy session involves a progressive set of prompts to initiate speech from the client, the therapy tool was set up with a progressive set of cue buttons which gradually give more and more assistance to the client in achieving a particular goal. For example, this might be help in pronouncing the consonant F. A set of six cue buttons mounted vertically on the right hand side of the screen allow for this positive reinforcement. These cues may take visual or spoken form. Another reason for a progressive interactive strategy is the problem of complexity which has plagued so many previous interface designs for the disabled [START_REF] Seidle | Is Home Care Less Expensive?[END_REF].

The sound cues may be both instructive and exemplary. One cue button will produce an instruction such as, "place the lower lip against your upper front teeth and blow" whilst another produces the target as required, the actual sound "fff ....". Another button links the consonant to a word so as to place the target sound in context. Yet another offers the word in the context of a sentence.

All cue buttons and command buttons have explanatory logos which appear at the bottom of the screen when the cursor moves over their area. There are over 300 digitised sound recordings associated with the therapy tool.

The system is normally controlled by a mouse and test models incorporate this means of user control. Other remote switch devices are currently being investigated. This will make the system more versatile and therefore suitable for quadriplegics [START_REF] Vanderheiden | Communication Systems and Their Components. Augmentative and Alternative Communication: American Speech and Language-Hearing Association[END_REF]. A system of this type should be flexible enough to adapt to suit the needs of each individual client and varying physical disabilities [START_REF] Reichle | Implementing Augmentative and Alternative Communication[END_REF]. The system being used by a speech therapy patient and therapist can be seen in Figure 1 below. Research by the author and therapists has established that phoneme synthesis is not suitable for the speech requirements of this system. A robotic sounding voice may be acceptable for certain games programs and simple communication tools but can never be adequate for the variety of accents, languages and key sounds that may be required by therapists. The Institution of Electrical and Electronics Engineers [START_REF]IEEE Recommended Practice For Speech Quality Measurements[END_REF], published a recommended practice for speech quality measurements, but this has never been fully adopted as the basis for standardised testing [START_REF] Edwards | Speech Synthesis: Technology for Disabled People[END_REF]. Edwards showed that no existing rule-based synthesiser came close to passing the test of being indistinguishable from natural speech.

To date, the system has been tested on a small pilot sample of patients in Australia. Initial tests at the Royal Perth Hospital have indicated that the response to the system for middle age stroke victims was very positive. Other age groups including children are included in forward planning for the later version. The interface has proved easy to understand and selected stroke patients have managed to control the system themselves, i.e. choosing and activating cue buttons.

Future tests will be coordinated to fine tune the system before full trials take place. It must be emphasized that full and robust trials have yet to be carried out. This is a work in progress. However, feedback from the first pilot tests highlighted the need for fast adaptability in design refinement which could be partly addressed by the digital ecosystem paradigm [START_REF] Chang | Digital Ecosystems and Comparison to Collaboration Environment[END_REF]. Issues such as universal access are also relevant [START_REF] Zimmermann | Prototype Implementations for a Universal Remote Console Specification[END_REF].

6

Other Support Clusters Speech therapy tools form just part of a wider picture of related hardware or software based support systems, all falling roughly within the catchment area of assistive technologies for the disabled. Most of these systems require special interface design and may cater for temporary or permanent loss of speech. The software tool described above is assisting in a hopefully temporary loss of speech. However, this loss may also be permanent. In which case alternative and augmentative devices such as portable speech aids (producing synthetically generated speech) are required. Even though these tools are closely related and familiar to most speech therapists, there is a huge gulf between the teams that produce and support both solutions. This communication gulf turns into a chasm when more diverse assisitive technologies are examined. These may include sensory loss such as hearing or visual impairment. The author has been working on both the above projects involving speech therapy and a substitution aid for natural speech. More recently we have been developing a navigation device for the blind. And this is where the infrastructure support clusters become very different.

There are numerous mobility aids and orientation mapping devices for the visually impaired on the market at present, some with significant drawbacks. Many assistive technology devices use ultrasonic pulse-echo techniques to gauge subject to object distance. Some use infrared light transceivers or laser technology to locate and warn of obstacles. These devices exhibit a number of problems, the most significant of which are related to the interface display that conveys navigation/obstacle warning information to the user. Other sensory channels should not be compromised by the device. This is exactly what can happen when, for example, audio signals are used in obstacle warning on/off displays.

The DEBII team has developed a prototype device, which it is hoped, will be the first step in addressing some of the above listed problems. Patent searches indicate this working prototype has a unique tactile interface design which, even in its basic form, should have distinct user advantages. As with some of the sonar systems listed above in the paper, this first prototype is best suited to outdoor use. Future models are not limited to sonar technology, however.

The design criteria has and will in the future, concentrate on intuitive interfaces that do not compromise certain other all-important sensory channels. These interfaces, now under development, will be configurable for users who are both deaf and visually impaired. There will also be an emphasis on ease of learning and use. It is unacceptable to expect someone, who may have multiple disabilities, to undertake a long and complex learning program in how to use a new device. The Innovation in the author's mobility aid design may be summarized as follows: 'A portable mobility aid incorporating warning obstacle ahead information with mapping capabilities as a stand-alone device.' At this stage, no further technical specification can be given due to IP novelty protection. It is hoped that in future papers, we will be able to concentrate more freely on the detailed technical aspects of the design. However, the current system uses ultrasound for range-finding and is mounted on a cane for test purposes. Initial tests have proved the system to be at least as effective as many of the alternative commercial systems available.

Digital Ecosystem Models

Issues of complexity with respect to individual requirements must be seen within the context of a wider ecology of the particular user, with that person clearly at the centre, contributing to a team solution. An established and highly successful ecological approach to designing individualized education programs for the disabled student has been refined over twenty years into a highly recommended model and is now regarded as 'best practice' [START_REF] Rainforth | Collaborative Teams for Students With Severe Disabilities[END_REF].

This ecological approach has not as yet permeated all areas of disability support. However, the power of the digital ecosystem framework is now accepted within many other disciplines, particularly with respect to small enterprise collaboration [START_REF] Chang | Digital Ecosystems and Comparison to Collaboration Environment[END_REF].

Within small business, the advent of the web has allowed sales penetration over vast distances. Accompanying these advances have come new modes of marketing and partnership possibilities that would have been impossible only a few years ago.

With this connectivity has come a fertile and dynamic business theatre that cannot be avoided if small enterprises are to survive. This interaction has led to collaborative workflow models [START_REF] Pudhota | Modelling the Dynamic Relationships between Workflow Components[END_REF].

The logic behind collaborative workflows is to produce a sequence of activities that not only produce a meaningful result, but also to facilitate small groups working together to achieve common goals. The actual physical distance and associated limitations between these entities then becomes less important as web based tools are used to link enterprises and their common aspirations [START_REF] Neumann | An Introduction to Web Objects[END_REF]. The entities themselves may be small companies competing against large predator corporations, or widely dispersed cottage industries (such as those associated with assistive devices) with a common interest [START_REF] Ulieru | The Holonic enterprise: a model for Internet-enabled Global Manufacturing Supply Chain and workflow Management[END_REF].

Beyond the standard empowerment the digital ecosystem model has provided, are more specific areas that are pertinent to such groups operating in harmony. One of the most important of these is trust evaluation [START_REF] Chang | Trust and Reputation for service-oriented Environments: Technology for building Business intelligence and Consumer Confidence[END_REF]. Other typical support areas are logistics and privacy [START_REF] Clark | Web Services Business Strategies and Architectures[END_REF][START_REF] Skinner | A Projection of the Future Effects of Quantum Computation on Information Privacy and Information Security[END_REF]. These would act as foundations for the framework l that is proposed.

Digital Ecosystems For Assistive Technology clusters (DEFAT) is a proposed collaborative cluster-based ecosystem model, neither limited by distance between clusters nor the particular disability types associated with each of the clusters. Individual clusters may include a range of specialist personnel associated with the support of a client requirement. The output of such an environment would not only be the efficient research and development of appropriate assistive devices, but also result in more streamlining for the teams in their everyday support of an individual, be that speech therapy for dysarthria patients or training in the use of a long cane or mobility aid for the visually impaired.

DEFAT Structure

With each client representing a nucleus at the centre of his or her support cluster, an individual's local ecological environment has been acknowledged (as discussed and cited in previous sections) as a worthwhile starting point, offering a framework from which specialist support action may be fleshed out.

Each support cluster would have a number of clients within its particular category of disability. Cluster membership would not be determined by distance or physical boundaries. The aim would be to maximize use of the digital ecosystem paradigm in order to break existing physical boundaries. By applying a DEFAT strategy, current digital technologies such as mobile, the internet and video conferencing can be coordinated and optimized to deliver the best outcome for all members of this ecosystem.

Open-ended but novel design solutions would be encouraged from both hardware and software developers. The sharing and exchange of common modular solutions at both a functional and user interface level would be part of the ecosystem membership requirement.

The difference would be in the focus and modular consideration of appropriate novel and relevant ideas, when first considering I.P. matters. This will not always be relevant to designs, but when it is, it should in fact enhance the potential for success within the DEFAT community itself, as well as in a wider context (external to the ecosystem).

Those academic cluster members who currently work within a limited research environment with a very small interest group would have the opportunity to share their research and ongoing projects on a wider stage within the digital ecosystem. Cross-disciplinary interaction would be nurtured by DEFAT.

A cluster of people with a vast range of interdisciplinary skills would focus on a user group of people all with a common disability. There would be many separate clusters, meeting the challenges of specific needs of different disability groups. As now, it may be assumed that special education specialists, therapists, medics, academics, engineers and particularly hardware and software experts would form part of each cluster, the main difference being a recognition of the greater ecosystem in which each cluster coexists and operates.

Users at the center of each cluster, the nucleus, would determine the nature of the environment. Clusters would communicate with each other for a common good and the ecosystem itself would be able to benefit from its size in terms of external links and its critical mass.

A starting point for such a structure may take into account the problem as defined by Liu et al when referring to building the right systems and the need for better tools and environments in their paper on component-based medical and assistive devices and systems [START_REF] Liu | Component-based Medical and Assistive Devices and Systems[END_REF]. They put forward a ten-year roadmap, which fits well as a start to implementing the DEFAT paradigm.

Clusters need to be client centered, taking into account breakthrough research such as that of Bach-Y-Rita into sensory substitution [START_REF] Bach | Sensory Substitution and the Human-Machine Interface[END_REF] and Merzenich into brain plasticity [START_REF] Mjenkins | Memory Concepts[END_REF].

A global advantage and DEFAT's greater mass would benefit the ecology on many levels. There would be lower manufacturing costs than is now associated with small-run dedicated systems production. This advantage would result from greater demand for DEFAT modular units across clusters and existing boundaries. Relatively large production runs catering for a global DEFAT module demand would drive production costs down.

Conclusion

The Digital Ecosystem and Business Intelligence Institute at Curtin University in Western Australia is developing a multimedia-based training tool for use by speech therapists and their clients. A more advanced system is currently under development. This new version is specifically aimed at people with motor speech disorders. It is believed that the design may, in time, be developed into a wider range of solutions which may incorporate related therapy requirements.

The author compares the development of the above speech therapy and speech aid solutions with other assistive device development examples; specifically those with sensory impairments, such as blindness. As most of these software and dedicated hardware solutions are produced by small, unlisted companies, there is little in the way of publicly available, reliable sales figures, and as such the addressable market success is not well defined. However, interviews conducted with industry experts, in addition to the small size of the companies themselves, suggest that these competing devices have so far failed to achieve any significant market presence, and in many cases, have inherent and significant user interface design issues. The author's prototype programme and proposed DEFAT framework development will, hopefully, be a step in the right direction.

Fig. 1 .

 1 Fig. 1. Speech therapy session with stroke patient.