Fedja Netjasov
email: f.netjasov@sf.bg.ac.rs

Andrija Vidosavljevic
email: a.vidosavljevic@sf.bg.ac.rs

Vojin Tosic
email: v.tosic@sf.bg.ac.rs

Henk Blom
email: blom@nlr.nl

Traffic Management Research and Development Seminar (ATM2011

Keywords: ACAS, Petri Nets, Safety Risk Assessment, Safety Critical Systems, Model Validation I

Current international regulations and policies do not consider the effect of airborne safety nets in the analysis of safety risks. This widely accepted practice tends to create significant tension between the realization of the ambitious safety improvement targets of SESAR and NextGEN, and standing regulations. In order to close this gap, there is a need for systematic development of safety risk analyses of airborne safety nets within an Air Traffic Management context. The aim of the research described in this paper is to address the systematic validation of an unambiguous mathematical model of Airborne Collision Avoidance System (ACAS) operations, together with its interactions with own and other pilots and with air traffic controllers. The specific modelling formalism used for this is Stochastically and Dynamically Coloured Petri Nets (SDCPN); which supports both mathematical analysis as well as Monte Carlo simulation. In order to build confidence, the focus of this paper is on the performance of a systematic validation of the developed model. This validation includes both comparisons against "real data" and comparison with the results of Eurocontrol's ACAS simulation model. Initial application of this validation process to the novel model shows that it is at least as good as the existing ACAS simulation model. However, the added value is that the novel model defines both an unambiguous mathematical model as well as an unambiguous simulation model.

INTRODUCTION

Airborne Collision Avoidance System (ACAS) constitutes a world-wide accepted last-resort means of reducing the risk of mid-air collision (MAC) between aircraft [1]. Key elements of the current ACAS consist of Traffic Alert and Collision Avoidance System (TCAS) II version 7 and procedures for pilots using this system. TCAS is intended to provide lastminute collision avoidance guidance directly to the flight crew [START_REF] Kuchar | The Traffic Alert and Collision Avoidance System[END_REF]. Hence, TCAS forms the last layer in the multi-layered defence against MAC, with all other layers typically belonging to ground based Air Traffic Management (ATM). Although recent accidents [START_REF] Brooker | The Uberlingen accident: Macro-level safety lessons[END_REF][START_REF] De Carvalho | Normal people working in normal organizations with normal equipment: System safety and cognition in mid-air collision[END_REF] show that the current ACAS is not perfect, there are many more known examples where ACAS made a positive difference.

Current ICAO risk/safety assessment policy is restrictive relative to ACAS in the sense that maximum values for mid-air collision risk are defined under the explicit assumption that the effect of an airborne safety net is not considered. This is also the case with Eurocontrol policy, which states that safety nets in general (both airborne and ground) should not be taken into account in the risk/safety assessment process [START_REF] Brooker | Why the Eurocontrol Safety Regulation Commission Policy on Safety Nets and Risk Assessment is Wrong[END_REF][START_REF] Brooker | Airborne Collision Avoidance Systems and Air Traffic Management Safety[END_REF]. Unfortunately, this may imply that ACAS improvements by SESAR/NextGen are not properly valued, as a result of which regulaton may pose overly conservative safety requirements upon the non-ACAS part of ATM.

In view of the SESAR and NextGEN objectives of increasing both capacity and safety there simply is a need to conduct safety risk analysis of new operations, including ACAS behaviour. An example is the Airborne Separation Assurance System (ASAS) as one of the new concepts whose interaction with ACAS has proven to be important from both the procedural and the human factor aspects [START_REF] Abeloos | Potential co-operation between the TCAS and the ASAS[END_REF][START_REF] Ivanescu | Effect of aircraft self-merging in sequence on an airborne collision avoidance system[END_REF][START_REF] De Oliveira | Modelling and Estimation of Separation Criteria for Airborne Time-Based Spacing Operation[END_REF][START_REF] Blom | Safety Risk Simulation of an airborne self separation concept of operation[END_REF][START_REF] Blom | Simulated collision risk of an uncoordinated airborne self separation concept of operation[END_REF]. These examples clearly show that the only way to include ACAS in the safety assessment process is through the modelling of ACAS operations.

Modelling of ACAS operations has been the subject of research since the introduction of TCAS. Many different modelling approaches with different needs have since been identified. Several approaches have emerged for verification i.e. formal analysis of complex safety-critical systems such as TCAS: Finite State Machine approach [START_REF] Leveson | Requirements Specification for Process-Control Systems[END_REF], State Charts [13; 14] and Hybrid Automata [START_REF] Livadas | High-Level modeling and analysis of TCAS[END_REF]. In order to understand human behaviour related to TCAS, Causal analysis [START_REF] Ladkin | Causal Analysis of the ACAS/TCAS Sociotechnical System[END_REF], and Timed Knowledge-based modelling and analysis [START_REF] Küster-Filipe | Timed Knowledge-based Modelling and Analysis: On the Dependability of Socio-technical Systems[END_REF], are applied. The necessity to examine ACAS safety is followed by development of encounter models based on Fault Tree Analysis coupled with the Monte Carlo Simulation [START_REF] Kuchar | Safety Analysis Methodology for Unmanned Aerial Vehicle (UAV) Collision Avoidance Systems[END_REF][START_REF] Zeitlin | Modeling for UAS Collision Avoidance[END_REF][START_REF] Zeitlin | Collision Avoidance for Unmanned Aircraft: Proving the Safety Case[END_REF], by Markov processes coupled with Bayesian networks [START_REF] Kochenderfer | A Comprehensive Aircraft Encounter Model of the National Airspace System[END_REF][START_REF] Kochenderfer | Airspace Encounter Models for Conventional and Unconventional Aircraft[END_REF] and Bayesian belief networks alone [START_REF] Neil | Modelling an Air Traffic Control Environment Using Bayesian Belief Networks[END_REF]. Finally, a range of encounter models is developed and applied with aim to study TCAS performances and evaluate a new TCAS logics or proposed changes of TCAS logic in use [START_REF] Drumm | Lincoln Laboatory Evaluation of TCAS II Logic Version 6.04a[END_REF][START_REF] Chludzinski | Lincoln Laboratory Evaluation of TCAS II Logic Version 7 -Volume I[END_REF][START_REF] Billingsley | TCAS Multiple Threat Encounter Analysis[END_REF][START_REF] Chludzinski | Evaluation of TCAS II Version 7.1 Using the FAA Fast-Time Encounter Generator Model[END_REF][START_REF] Espindle | Safety Analysis of Upgrading to TCAS Version 7[END_REF][START_REF] Temizer | Unmanned Aircraft Collision Avoidance Using Partially Observable Markov Decision Processes[END_REF][START_REF] Kochenderfer | Model-Based Optimization of Airborne Collision Avoidance Logic[END_REF]. Complementary to these mathematical models, an interactive simulator InCAS was also developed [START_REF] Gain | Guide to Methods and Tools for Safety Analysis in Air Traffic Management[END_REF][START_REF]InCAS 2.6 -User Guide[END_REF] in order to replay and analyse ACAS related incidents and to learn from encounters; and a tool called Replay Interface for TCAS Alerts (RITA) was developed for ACAS training of air traffic controllers and pilots [START_REF] Gain | Guide to Methods and Tools for Safety Analysis in Air Traffic Management[END_REF].

Recently [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF][START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF] have developed another model of ACAS using a Petri net formalism that is named Stochastically and Dynamically Coloured Petri Nets (SDCPN). This SDCPNbased ACAS model covers TCAS II version 7 as well as the pilots, the air traffic controllers, some other relevant equipment and the interactions between these model entities. The main reason for using the SDCPN is the possibility of modelling complex relations existing between different system elements (humans, procedures, equipment) in a systematic and compositional way [START_REF] Everdij | Compositional Specification of a Multi-Agent System by Stochastically and Dynamically Coloured Petri Nets[END_REF]. This also allows for new ACAS models, for some previously or future developed SDCPN modules related to current or advanced operational concepts, to be added. This is confirmed by previous experiences using Petri Nets for safety analysis [START_REF] Leveson | Safety Analysis Using Petri Nets[END_REF], as well as Dynamically Coloured Petri Net (DCPN) for aviation purposes [START_REF] Shortle | Simulating Collision Probabilities of Landing Airplanes at Non-towered Airports[END_REF][START_REF] Everdij | Modelling Lateral Spacing and Separation for Airborne Separation Assurance Using Petri Nets[END_REF][START_REF] Netjasov | A Review of Research on Risk and Safety Modelling in Civil Aviation[END_REF]. Moreover, the SDCPN formalism brings both Monte Carlo simulation approach as well as powerful analysis frameworks within reach [START_REF] Everdij | Enhancing Hybrid State Petri Nets with the Analysis Power of Stochastic Hybrid Processes[END_REF], and it is fully embedded in the advanced safety risk assessment methodology TOPAZ [START_REF] Everdij | Petri Nets and Hybrid State Markov Processes in a Power-Hierarchy of Dependability Models[END_REF][START_REF] Everdij | Modelling Hybrid State Markov Processes Through Dynamically and Stochastically And Dynamically Coloured Petri Nets[END_REF][START_REF] Everdij | Piecewise deterministic Markov processes represented by dynamically coloured Petri nets[END_REF].

Hence, the SDCPN-based ACAS model from [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF] has a significant advantage over other ACAS models when it comes to mathematical analysis and flexibility to integrate the model with other elements of an ATM operation. But what happens to the validity of the SDCPN-based ACAS model? A well-known fear is that a model with mathematical and flexibility advantages easily leads to a model that cannot be validated. The aim of the current paper is to investigate this validation aspect of the SDCPN-based ACAS model.

In order to accomplish this, we developed a systematic validation process based on the model validation principles that have been developed by [START_REF] Sargent | Verification and Validation of Simulation Models[END_REF][START_REF] Balci | Verification, Validation and Testing[END_REF]. Their validation principles are based on viewing model validation as a "substantiation that a model within its domain of applicability possesses a satisfactory range of accuracy consistent with the intended application of the model". In [START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF] these principles have been used for the development of a systematic validation approach of the SDCPN-based ACAS model. Subsequently [START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF] has applied the developed validation approach to the model in [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF]. The findings of [START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF] were twofold: the developed validation approach worked well, and at the same time the outcomes revealed that there were some issues that could be improved in the conceptual ACAS model of [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF] only.

The aim of the current paper is threefold:

-To outline the improved conceptual ACAS model, including an explanation of the changes relative to [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF];

-To develop a systematic validation process according to the principles of [START_REF] Sargent | Verification and Validation of Simulation Models[END_REF][START_REF] Balci | Verification, Validation and Testing[END_REF], using both real life encounter data and Eurocontrol's ACAS simulation model;

-To initially apply this validation process to the newlydeveloped ACAS model.

This paper is organized as follows. Section II provides a detailed description of the ACAS conceptual model, including an explanation of the improvements made over the conceptual model in [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF]. Sections III develop the systematic validation approach. Section IV presents the initial validation results obtained for the novel ACAS model. Section V draws conclusions.

II. ACAS CONCEPTUAL MODEL

Since January 2005, ICAO mandates the use of ACAS worldwide for all aircraft with more than 19 passenger seats or with a maximum take-off weight exceeding 5,700 kg. TCAS II Version 7 is the only TCAS version that complies with ICAO Standards and Recommended Practices (SARPs) for ACAS [1,[START_REF] Kuchar | The Traffic Alert and Collision Avoidance System[END_REF][START_REF]Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System II (TCAS II) Airborne Equipment -Volume I[END_REF][START_REF] Banks | Introduction to TCAS II Version 7[END_REF]. TCAS is designed to work autonomously, i.e. without support of the aircraft navigation equipment, and independently of the ground systems used to provide Air Traffic Control (ATC) Services [START_REF] Banks | Introduction to TCAS II Version 7[END_REF]. Generally, TCAS monitors the airspace around the own aircraft and warns pilots of the presence of other aircraft, so called intruders, which may present a MAC threat. A crucial part of TCAS is a Collision Avoidance Logic, the main functions of which are [START_REF] Banks | Introduction to TCAS II Version 7[END_REF]: tracking, traffic advisory, threat detection, resolution advisory, TCAS/TCAS coordination, advisory annunciation and performance monitoring.

In order to model an ACAS operation in this research, the operation is divided into the following phases [1, [START_REF]Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System II (TCAS II) Airborne Equipment -Volume I[END_REF][START_REF] Banks | Introduction to TCAS II Version 7[END_REF]: Normal flight, Appearance of Traffic Alert (TA), Appearance of Resolution Advisory (RA) and Return to normal flight. Details about each phase are given in [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF][START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF].

The ACAS conceptual model is based on [1] and [START_REF] Banks | Introduction to TCAS II Version 7[END_REF] documents, and specifies mathematical models of all algorithms used for threat detection and threat resolution.

A. Threat detection algorithms

In order to determine whether a collision threat exists, i.e. to issue a TA or an RA, both the range and vertical criteria must be satisfied; i.e. if one of them is not satisfied, TCAS will not issue a TA or an RA. For checking whether the range and vertical criteria are satisfied, Range tests and Altitude tests are constantly performed during an encounter. Criteria used for making a decision about TA and RA issuance depend on the Sensitivity Level (SL). SL depends of the aircraft altitude range and contains values for horizontal and vertical τ thresholds in case of TA or RA issuance, dimensions for protected airspace (Distance Modification -DMOD and Altitude Limit -ALIM) which should be satisfied in case of slow closure encounters when τ threshold values are not appropriate and vertical threshold value (ZTHR) at Closest Point of Approach (CPA). During an encounter, if the horizontal or vertical τ is lower than the TA threshold or if the horizontal and vertical miss distance is lower than the TA DMOD and TA ALIM respectively, then a TA is annunciated. If the situation further worsens and τ values are lower than the RA threshold or if the miss distances are lower than the RA DMOD and RA ALIM respectively, or predicted vertical miss distance at CPA is lower then ZTHR, then an RA is annunciated [1,[START_REF] Banks | Introduction to TCAS II Version 7[END_REF].

For the purpose of range and altitude tests, aircraft are identified in a Cartesian coordinate system. Let x i t = (x i x,t , x i y,t , x i z,t ) T and v i t = (v i x,t , v i y,t , v i z,t ) T be the 3D position and 3D velocity of aircraft i; the superscripts x and y refer to the horizontal axis system, and z stands for the altitude. Let θ i t represent an orientation velocity vector v i t in the horizontal plane (measured from the x axis in counter-clockwise direction, where 0 ≤θ i t ≤ 2π) and let ψ i t represent the orientation of velocity vector vit in the vertical plane (measured from the horizontal plane up as positive and down as negative, whereπ/2 ≤ψ i t ≤ π/2). Let x ik t = x i tx k t be the distance in 3D space between own aircraft i and intruder aircraft k at time t and let v ik t = v i tv k t be the relative velocity (closing speed) between aircraft at time t. a) Range test: At each moment t, both the distance and the relative velocity between own and intruder aircraft in the horizontal plane are calculated. Knowledge about both values is required in order to calculate the "time to closest point of approach" (in horizontal direction, i.e. the range τ). The encounter geometry observed in the range test is shown in Figure 1.

Let x i h,t = (x i x,
( ) ik t z ik t z ik t z v x , , , / - = τ (5)

c) TA or RA issuance

The Range and Altitude tests compare given criteria and calculated values for

τ ik h,t , τ ik z,t , ik t h, x ˆ, ik t z, x ˆ, ik t h, k i t h x , , τ + and ik t z, ik t h x , τ + . If the TA ik t h, DMOD x k i t h < + , , τ
(horizontal miss distance filter), then an intruder aircraft is declared to be a threat to the own aircraft and then the TA and RA issuance process follows. So, according to [1], TA alert shall be issued whenever one of the following sets of conditions is satisfied:

( ) ( ) TA ik t z, TA ik t h, τ τ 0 τ τ 0 < < ∧ < < ∨ ( ) (
)

TA ik t z, TA ik t h, LIM A x τ τ 0 < ∧ < < ˆ ∨ ( ) ( ) TA ik t z, TA ik t h, τ τ 0 MOD D x < < ∧ < ˆ ∨ ( ) ( ) TA ik t z, TA ik t h, LIM A x MOD D x < ∧ < ˆ ∨ ( )       < ∧ < + TA ik t h, TA ik t z, DMOD x LIM A x k i t h , , ˆτ ∨ ( )       < ∧ < < + TA ik t h, TA ik t z, DMOD x τ τ 0 k i t h , , τ (6)
And RA will be issued when one of the following set of conditions is satisfied:

( ) ( )       < ∧ < ∧ < < + ZTHR x LIM A x τ τ 0 ik t z, RA ik t z, RA ik t h, k i t h , , τ ∨ ( ) ( )       < ∧ < < ∧ < < + ZTHR x τ τ 0 τ τ 0 ik t z, RA ik t z, RA ik t h, k i t h , , τ ∨ ( ) ( ) ( ) ik t h, ik t z, RA ik t z, RA ik t h, τ τ τ τ 0 τ τ 0 < ∧ < < ∧ < < ∨ ( ) (
)

RA ik t z, RA ik t h, LIM A x τ τ 0 < ∧ < < (7)
OR the same set of conditions where the equation ( )

RA ik t h, τ τ 0 < < is replaced with the equation ( ) RA ik t h, DMOD x < ˆ,
producing the set of conditions [START_REF] Ivanescu | Effect of aircraft self-merging in sequence on an airborne collision avoidance system[END_REF], or with the equation

      < + RA ik t h, DMOD x k i t h , ,
τ producing the set of conditions (9).

In the above, ik t z,

k i t h x , , τ + is defined as follows: if RA ik t h, τ τ <
at moment t then predicted vertical positions of aircraft i and k at CPA are given by the following equations:

ik t h i t z i t z i t z, v x x ik t h , , , , τ τ ⋅ - = + (10) ik t h k t z k t z k t z, v x x ik t h , , , , τ τ ⋅ - = + (11)
Predicted vertical miss distance between aircraft i and k at CPA is given by the following equation:

k t z, i t z, ik t z, ik t h ik t h ik t h x x x , , , τ τ τ + + + - = (12) Similarly, ik t h, k i t h x , , τ + is defined as follows: if TA ik t h, τ τ <
at moment t then predicted horizontal positions of aircraft i and k at CPA are given by the following equations:

ik t h i t h i t h i t h, v x x ik t h , , , , τ τ ⋅ + = + (13) ik t h k t h k t h k t h, v x x ik t h , , , , τ τ
⋅ + = + (14)
Predicted horizontal miss distance between aircraft i and k at CPA is given by the following equation:

k t h, i t h, ik t h, ik t h ik t h ik t h x x x , , , τ τ τ + + + - = (15)

B. Threat resolution algorithm

Once a threat is identified, a two-step process is followed to select the appropriate RA for the given encounter geometry. In the first step an appropriate sense is selected (upward or downward); that is, whether the aircraft needs to climb or to descend. In the second step an appropriate strength (vertical speed) is determined; that is, how rapidly the aircraft needs to change its altitude.

a) Sense selection

Let t be the moment at which an RA for own aircraft i is issued, i.e. τ RA seconds remain until CPA with intruder aircraft k. The TCAS Logic makes trials with upward and downward sense for own aircraft, in order to determine which sense provides the most vertical separation at CPA (time moment t+τ RA in Figure 2) under the assumption that intruder aircraft doesn't change its flight profile. The sense which provides the greatest vertical separation shall be selected. Consider a possible vertical position of aircraft i at moment t+τ RA during the trial (see Figure 2):

• if the upward sense is selected RA i t z i t z i t z i t z v x up x RA τ τ ⋅ ∆ + + = + ) ( ) ( , , , , (16)
• if the current rate is maintained

RA i t z i t z i t z v x current x RA τ τ ⋅ + = + , , , ) ( (17)
• if the downward sense is selected

RA i t z i t z i t z i t z v x down x RA τ τ ⋅ ∆ - + = + ) ( ) ( , , , , (18)
where ∆ i z,t has a fixed value of 1500 feet/min [START_REF] Kuchar | The Traffic Alert and Collision Avoidance System[END_REF][START_REF]Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System II (TCAS II) Airborne Equipment -Volume I[END_REF].

t+τRA i k a < ALIM RA b Upward Downward ) ( , current x i t z RA τ + x ik z,t ) ( , up x i t z RA τ + ) ( , down x i t z RA τ + Figure 2. RA sense selection (illustrative example)
Two vertical separations at CPA between own aircraft i and intruder k, are recognized in the sense selection process and are given by the following equations (see Figure 2):

) ( ) ( , , current x up x a k t z i t z RA RA τ τ + + - ≡ (19)
) ( ) ( , , current x down x b k t z i t z RA RA τ τ + + - ≡ ( 20
)
The sense is represented by the variable c i t which assumes the value: c i t = 1 in case of the upward sense selected, c i t = -1 in case of downward sense, and c i t = 0 otherwise. In case aircraft i already receives a sense from aircraft k before it has finished its own sense calculations then:

c i t = -c k t ( 21
)
otherwise:

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )                         ≥ ∧       = ∈ ∃ ∧ > ∨ < ∧ ≤        ≥ ∧       = ∈ ∃ ∧ ≤ ∨ < ∧ > - = + + therwise o 0, ALIM a 0 x that such a b ALIM b a b if 1, ALIM b 0 x that such a b ALIM a a b if 1, c RA ik t z, RA RA RA ik t z, RA RA i t ε ε τ ε τ ε ] , 0 ( ] , 0 ( (22)
The obtained sense for the own aircraft i is coordinated through the Mode S data link with intruder aircraft k with the aim to avoid that both aircraft select the same vertical sense.

Hence, the RA sense sent to the intruder aircraft satisfies:

     - = = = otherwise , c 0 c or 0 v if 0, c i t i t k t z, k t (23)
Equation ( 23) covers the possibility that

c k t = 0 if 0 v k t z, = .
This means that when the intruder aircraft is in horizontal (level) flight, then its flight profile will not be changed, i.e. it will not receive an RA.

b) Strength selection

Once the sense has been selected, TCAS logic will determine RA strength. RA strength should be least disruptive to the existing flight path, while providing at least ALIM RA vertical separation between aircraft i and k at CPA (time moment t+τ RA ), under the assumption that intruder aircraft doesn't change the flight profile. This means that the change of vertical speed ∆ i* z,t should be minimal. The determination of the appropriate strength (vertical speed) is carried out as follows:

if RA ik t z ALIM current x RA ≥ + ) ( , τ
then no RA is issued, otherwise the strength is calculated using:

     = ∆ - = ∆ + = 1 - c ; v 1 c ; v v i t * i t z, i t z, i t * i t z, i t z, * i t z, ˆ (24)
where:

( ) ( ) RA RA i t z i t z RA k t z k t z RA i t z v x v x ALIM τ τ τ / ˆ, , , , * ,       ⋅ + - ⋅ + + = ∆ (25)
c) RA modification Nine seconds [START_REF] Banks | Introduction to TCAS II Version 7[END_REF] after an RA has been issued, i.e. at moment t+9, logic is checking the evolution of the encounter. At that moment τ RA -9 second remains until CPA.

In the case that at moment t+9 the predicted vertical separation between aircraft i and k at CPA is

RA ik t z ALIM x RA ≥ +τ ,
, then RA should be modified. The new RA could contain a sense modification and/or strength modification. Modified sense values are then given by the following equations:

     ≥ - = + + otherwise , c ALIM x , c c i t RA ik τ t z, i t i t RA 9 (26)      - = = = + + + + otherwise , c 0 c or 0 v if 0, c i 9 t i 9 t k 9 t z, k t 9 (27)
and modified strength value by the following equation:

     = ∆ - = ∆ + = + + + + + + + 1 - c if ; v 1 c if ; v v i 9 t * i 9 t z, i 9 t z, i 9 t * i 9 t z, i 9 t z, * i 9 t z, ˆˆ (28)
where:

( ) (
)

) 9 /( ) 9 ( ˆ) 9 ( ˆ9 , 9 , 9 , 9 , * 9 , -         - ⋅ + - - - ⋅ + + = ∆ + + + + + RA RA i t z i t z RA k t z k t z RA i t z v x v x ALIM τ τ τ (29)

d) Clear of Conflict annunciation

The following conditions should be satisfied in order to announce CoC and terminate the encounter: a) RAs may terminate for a number of reasons: normally, when the conflict has been resolved and the threat is diverging in range [1, 13]; b) a CoC occurs after an encounter has been resolved [START_REF]Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System II (TCAS II) Airborne Equipment -Volume I[END_REF]).

Let t CPA be the moment when both aircraft are at CPA. Let t' > t CPA be the first moment when both aircraft are safely passing the CPA and the following condition is satisfied:

CPA ik t h, ik t' h, x x > (30)
then "Clear of Conflict" will be annunciated and the TCAS encounter is terminated.

C. Enhancements over the conceptual model in [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF] In [START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF] initial SDCPN-based ACAS model from [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF] has been partially evaluated using the validation approach described in the sequel of this paper. Based on these outcomes some issues have been identified for which this initial model should be improved. This has resulted in the ACAS model as described in the above equations (1) to [START_REF] Kochenderfer | Model-Based Optimization of Airborne Collision Avoidance Logic[END_REF]. These equations differ from those in [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF] as follows:

• αβ tracking has been included, as a consequence in all conceptual model equations, (1) through [START_REF] Kochenderfer | Model-Based Optimization of Airborne Collision Avoidance Logic[END_REF], true aircraft states have been replaced by estimated aircraft states; • an improved set of conditions for triggering TA and RA have been introduced in equations ( 6) to [START_REF] Livadas | High-Level modeling and analysis of TCAS[END_REF] including also horizontal miss distance filter; • modified conditions for own aircraft sense selection in equation ( 22); • an additional condition for determining intruder sense has been introduced in equation ( 23); and • an RA modification has been introduced in equations ( 26) through [START_REF] Temizer | Unmanned Aircraft Collision Avoidance Using Partially Observable Markov Decision Processes[END_REF].

III. SYSTEMATIC VALIDATION PROCESS

Proper validation of the developed SDCPN-based model of ACAS operations is a prerequisite in order to establish confidence in it for safety risk analysis purposes. In [START_REF] Sargent | Verification and Validation of Simulation Models[END_REF] and [START_REF] Balci | Verification, Validation and Testing[END_REF] model validation is defined as "substantiation that a model within its domain of applicability possesses a satisfactory range of accuracy consistent with the intended application of the model". Following [START_REF] Balci | Verification, Validation and Testing[END_REF] the main principles of validation are:

• Validation must be conducted throughout the entire life cycle of a simulation study; • The outcome of simulation model should not be considered as a binary variable where the model is absolutely correct or incorrect. • A simulation model is built with respect to study objectives and its credibility is judged with respect to those objectives. Since a model is an abstraction of a system, perfect representation is never expected. The outcome of the model validation should be considered as a degree of credibility on a scale from 0 (absolutely incorrect) to 100 (absolutely correct) [START_REF] Balci | Verification, Validation and Testing[END_REF]. Sargent in [START_REF] Sargent | Verification and Validation of Simulation Models[END_REF] presents "validation techniques and tests".

Among the numerous techniques, those accepted in this research, i.e. recognized as best suitable for the available data, are the following:

• Historical data validation: if historical data exists it is used to determine (test) whether the model behaves as the system does.

• Comparison to other models: various outputs of the simulation model being validated are compared to outputs of other simulation models that have been validated.

A. Validation Process of ACAS Operations Model

The aim of validation in this research is to provide evidence how well the model represents real world ACAS operation, taking into account that the model is developed for the purpose of risk and safety assessment.

In order to validate the developed model, an iterative validation process is proposed in this research based on abovementioned thinking, were in each iteration the developed model is improved if necessary, and if not it passes to the next iteration. It consists of four successive validation levels, i.e. iterations, were each level is represented with a certain question, while successive levels become more detailed. The following questions are asked (Figure 3) [START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF][START_REF] Netjasov | Validation of SDCPN Model for TCAS II Version 7 Operation in ATM[END_REF]:

• At Level 1 -Is TCAS activated?

• At Level 2 -Are the same TCAS events occurring?

• At Level 3 -If RA occurs, is the resolution manoeuvre similar?

• At Level 4 -Are corresponding horizontal and vertical separations at CPA close enough to the same separation in Control case?

At each validation level a modelled case (encounter) is compared with Control case (which could be from real life or from other model). If at one level the validation results are not satisfactory, then a modification of model might be proposed. If the validation results are satisfactory then it is possible to pass on the next validation level. The process is repeating until the end of validation is reached.

B. Collection of ACAS encounter data

The approach taken in this research is to make validation by comparing the outputs from the developed SDCPN model with Real Life Encounters or outputs from other models with similar purpose. Inputs for the comparison are data for seven real life encounters, chosen in a way to represent different conflict situations, provided by Technical University Braunschweig [START_REF] Gottstein | Five Million Flight Hours Continuous Reception of ACAS -Communications and Reporting of ACAS/TCAS -Interventions in the German Airspace[END_REF]. A Eurocontrol model InCAS was available and was used for validation of SDCPN model [START_REF] Gain | Guide to Methods and Tools for Safety Analysis in Air Traffic Management[END_REF][START_REF]InCAS 2.6 -User Guide[END_REF][START_REF] Dean | TCAS Analysis of Safety events using InCAS[END_REF]. This model was well-proven across Europe in TCAS encounters analysis (see Table I for comparison with SDCPN model). Real Life data served as a basis for the preparation (reconstruction and approximation) of input data for both the SDCPN-based model of ACAS operations as well as the InCAS model (Figure 4). Although the number of real life encounters was not large, this kind of validation was very valuable due to the fact that it is rarely performed for such models. The following outputs were chosen for validation:

• selected sense and strength for the issued RA;

• minimum horizontal distance at CPA as well as corresponding vertical distance and time;

The facts that TCAS is activated and RA is issued are considered at validation Level 1 and 2 respectively. Further details from the data are used at validation Levels 3 and 4.

Real life encounters data serve to create inputs for the SDCPN-based model of ACAS operations and InCAS model. Namely, the horizontal situation and horizontal distance versus time plots are used for aircraft initial positions and magnetic headings. Positions are given in the geographical coordinate system. They are read directly from the plots and translated into the Cartesian coordinate system used in SDCPN-based and InCAS models.

Similarly, magnetic headings were calculated taking the coordinates for initial positions and positions in which the Clear of Conflict message was issued, and using basic equations from theory of navigation. Additionally, the time interval between the initial point and point were CoC was issued, allowed for the calculation of ground speed.

Aircraft initial altitudes are calculated using the vertical situation versus time plot. As was the case with headings, the rates of descend/climb is calculated using the vertical differences between altitudes at the aircraft initial position and position at which the RA message was issued, and time between those two positions.

All the variables determined by the previous method are used in the model having constant values, while in real life encounters they aren't constant. Using this input data SDCPNbased as well as InCAS model produces outputs which are used for validation, together with the real life data.

C. Comparison against Real Life

Taking into account the abovementioned, it is likely that certain differences between model outputs and real life cases will appear.

Possible causes of such differences are the following [START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF]:

•

Inputs for both ACAS SDCPN and InCAS models are obtained by reconstructing the real event. Such a procedure may generate certain errors in input values because it uses the average ground speed, rate of descent/climb and magnetic heading values. Additionally, a geographic coordinate system from reality was transformed into the Cartesian coordinate system used in both models. This transformation could form an additional source of errors in the aircraft position input;

•

RA strength in the SDCPN-based model is assumed to have the maximal value, from the suggested range of strength values (maximal from the flight profile change point of view). It is applicable both for initial RA issuance and for RA modification. Applied values in reality belong to the range suggested to pilots during specific encounter; InCAS applied values are not fully known however.

•

Certain differences in TCAS logic could also be a reason for differences, especially due to the possibility that aircraft in the real encounters were equipped with TCAS II equipment by different manufacturers, while in the SDCPNbased and InCAS model all aircraft are equipped with the same model of TCAS II Version 7. All manufacturers should satisfy minimum requirements for TCAS II, but it could be expected that some differences in logic might exist due to the fact that manufacturers respond to the required minimum providing even more rigorous logic. For example, the SDCPN and InCAS models are based on documents dating from 2007 and earlier [1, [START_REF]Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System II (TCAS II) Airborne Equipment -Volume I[END_REF][START_REF] Banks | Introduction to TCAS II Version 7[END_REF]. However, in 2008 a new set of documents was issued containing a new logic called Version 7.1 [START_REF]Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System II (TCAS II) Airborne Equipment -Volume I[END_REF][START_REF] Eurocae | Minimum Operational Performance Standards for Traffic Alert and Collision Avoidance System II (TCAS II) -Volume I[END_REF]. According to authors best knowledge these changes still haven't came into force [START_REF] Hindsight | TCAS II and Level Bust[END_REF]. However, in the near future there may be certified aircraft that carry an enhanced ACAS, while this is not yet covered by all models of ACAS.

IV. ACAS MODEL VALIDATION RESULTS

A. Validation -Level 1 results

At Level 1 a simple matrix (Table II) is constructed, where for each pair of Control and SDCPN-based cases, a frequency of occurrence, i.e. number of situations in which both cases are the answer to the given question, is presented.

It would be ideal that pairs of cases are located in the green fields, meaning that a perfect match exists between SDCPNbased model and Control for the given case. If the pair of cases are located in the red corner it means that something was disabling the TCAS to activate in the SDCPN-based model, while in the Control in the same case TCAS was activated. Such a situation is considered as unwanted from the safety assessment point of view as the model does not "recognize" appropriate severity of the case.

If cases are located in the yellow corner it means that the SDCPN-based model was enabling TCAS to activate in situations when in the Control TCAS was not activated for the same case. Such a situation is considered a false alarm, but it is not negative although it is not wanted from the safety assessment point of view, i.e. the model is conservative compared to the Control. While some of the cases are different Outputs Outputs ACAS SDCPN

Real life encounters

Reconstructed

Real life encounters

Outputs

InCAS

Reconstructed

Real life encounters

Historical Data Validation Historical Data Validation

Comparison with Other Models LEGEND: modelling/simulation validation or cases in red or yellow corner exist, it might be decided to improve the model as much as possible.

The results of Level 1 validation between SDCPN-based model of ACAS operations and Real Life data are presented in Table II. In 7 out of 7 available cases (encounters) ACAS was activated both in reality and in the SDCPN-based model.

The results are in green fields, meaning that satisfactory validation results were obtained. The same results are obtained in the case of validation between InCAS model and Real Life and SDCPN-based model and InCAS (that's why they are not presented in separate tables).

B. Validation -Level 2 results

If the answer to the Level 1 question is positive, then at Level 2 the question is more detailed. A matrix is constructed (Table III), where for each pair of Control cases and SDCPNbased cases, a frequency of occurrence, i.e. number of situations in which both cases are answer on the given question is presented. Situations that are covered in Table III are the following:

• No event -case in which there is no need for either TA or RA to be issued;

• TA -case in which a TA is issued;

• RA -case in which an RA is issued and satisfactory resolved (vertical separation at CPA appropriate);

• RA* -case in which an RA is issued but not satisfactory resolved (vertical separation at CPA violated);

• RA MAC -case in which an RA is issued but MAC occur.

Like before, green fields in the matrix present a matching situation, i.e. situations in which answers to the given question are the same both in the SDCPN-based model and in the Control, for the same case.

If the situation falls in the red fields (above and right from the green diagonal) this means that TCAS is not properly activated or not activated at all, and this behaviour of the SDCPN-based model is designated as unwanted from a safety assessment point of view, since the model does not "recognize" the appropriate severity of cases.

Finally, if the situations fall in the yellow filed (below and left from the green diagonal) this means that TCAS was activated in situation when in the Control for the same case TCAS was not activated or it is but it was less severe. Such a situation is considered a false alarm, but it is not negative although it is not wanted from the safety assessment point of view. As in the case of the previous validation level, it might be suggested to improve the model as much as possible if some of the results are different or cases in red or yellow corner exist. IIIa. It was shown that in 6 out of 7 available encounters TCAS generated a RA both in reality and in the SDCPN-based model (results are in the green fields, meaning that satisfactory validation results were obtained).

Level 2 validation results for comparison between ACAS SDCPN model and Real Life data are presented in Table

However, in 1 out of 7 encounters a TA only is issued in the SDCPN-based model instead of RA which was activated in reality, placing it in a red field. Additional analysis showed that trajectories of aircraft in the given case in reality were not converging or crossing, and the aircraft were laterally and vertically well separated when the RA was issued. Therefore there is a possibility that in reality this encounter was a false alarm.

Similar behaviour is observed in the case of comparison between InCAS model and Real Life data (Table IIIb). Here it is also shown that in 1 out of 7 encounters a TA only was issued in the InCAS model instead of RA which was activated in reality, placing it in a red field.

It happened that is the same encounter as it is in case of ACAS SDCPN model. That's why this result was taken as the necessary proof for the decision to pass on to the level 3 validation.

At validation level 2, the responses of InCAS and SDCPN model are the same. The comparison of the ACAS SDCPN model and InCAS model (Table IIIc) shows that perfect match exist, i.e. that all encounters fall on the green diagonal. If the answer to the Level 2 question is positive, then at Level 3 for events in which RA is activated the question takes into account the type of resolution manoeuvre (chosen resolution sense). A matrix should be constructed (Table IV), where for each pair of Control and SDCPN-based cases a frequency of occurrence, i.e. number of situations in which certain pair of senses, occurs.

Due to the possibility that a large number of different sense combinations can appear, it was decided to aggregate them into two groups of similar pairs of senses. Aggregation is always a difficult issue and the possibility always exists that some important information and differences between the model and control could remain hidden.

For each cell in the matrix in Table IV further division is made into pairs of senses, which can be "u/d" or "d/u", were first letter is related to own aircraft while the second to intruder. "u/d" presents situations in which the own aircraft obtains an up or up-level or no-change sense while intruder aircraft obtains a down or down-level or no-change sense. The similar applies for the "d/u" combination, were own aircraft receives down or down-level or no-change sense instructions, while the intruder receives the up or up-level or no-change sense. Here, ideally all situations should fall on the upper-leftlower-right diagonal, meaning that manoeuvres (senses) provided both by the Control and the SDCPN-based model are similar. If the situation does not fall onto it, this means that the corresponding manoeuvres (senses) differ.

Table IVa presents the results of the Level 3 validation in case of comparison between ACAS SDCPN model and Real Life data. It was shown that in 5 out of 6 available encounters (which were placed in green field at level 2) resolution manoeuvre, i.e. issued sense, was similar both in reality and in the SDCPN-based model.

In one out of 6 encounters the manoeuvre was different (see red number in Table IVa, it was actually encounter 2). However, it should be considered that there might be more than one solution for certain encounters. This could be the explanation for the differences that appear at this level.

To verify this, complementary Monte Carlo simulations for encounter 2 have been carried out. The input value for vertical speed of own aircraft is randomly changed in the range of ±10% of nominal value -1275 fpm, i.e. in the range [-1148, 1403 fpm] using the uniform probability distribution. The experiment is repeated 1000 times. In 129 encounters manoeuvre (sense) was the same like in reality with the resulting vertical separation at CPA being between 949 and 1543 ft.

Additionally, it was found that for vertical speed in the range of [-1148, -1185 fpm] the manoeuvre was the same as in reality, and in the range [-1186, -1403 fpm] the manoeuvre was opposite. This experiment demonstrated that the selected sense could easily switch to the opposite when small changes in vertical speed are introduced. Also, the aggregation of similar senses into two groups could be the potential reason for differences. Although certain differences exist, it seems that in encounters involving two aircraft such aggregation would be suitable, while in encounters with three or more aircraft involved, the chosen senses are very sensitive to encounter geometry. So, disaggregation of similar senses could be additionally considered for such encounters. In the case of InCAS vs. Real Life, all encounters fall on the green diagonal (Table IVb), meaning that the selected senses were similar.

Table IVc shows that in the case of ACAS SDCPN vs. InCAS there is still one (same) encounter as in Table IVa, were SDCPN model selected a different sense. This result also proves that there are some differences in the sense selection algorithms used in the SDCPN model relative to real life ACAS, or ACAS used in the InCAS model.

1 0 0 0 0 0 RA d/u 1 4 0 0 0 0 u/d 0 0 0 0 0 0 RA* d/u 0 0 0 0 0 0 u/d 0 0 0 0 0 0 SDCPN RAMAC d/u 0 0 0 0 0 0 b) Real Life Encounters Data u/d 2 0 0 0 0 0 RA d/u 0 4 0 0 0 0 u/d 0 0 0 0 0 0 RA* d/u 0 0 0 0 0 0 u/d 0 0 0 0 0 0 InCAS RAMAC d/u 0 0 0 0 0 0 c) InCAS u/d 1 0 0 0 0 0 RA d/u 1 4 0 0 0 0 u/d 0 0 0 0 0 0 RA* d/u 0 0 0 0 0 0 u/d 0 0 0 0 0 0 SDCPN RAMAC d/u 0 0 0 0 0 0 D. Validation -Level 4

results

The idea of the validation process in this research is to go into greater detail when we approach deeper levels of validation (from rough to fine information). That is why at Level 4 a graphical comparison between horizontal and vertical separations at CPA following implementation of the issued RA was chosen instead of table, i.e. continuous rather than discrete variables. Usage of a table is avoided due to necessity to perform certain aggregations, which will hide some of the results that are visible in graphical form.

The perfect case would be in situation where both SDCPN and Control data lay on the line, presented with equation x = y, meaning that both the Control and the SDCPN-based model have the same horizontal and vertical separation at CPA. Under Level 4 validation, the correspondence between ACAS SDCPN-based model data and Real encounter data are presented in Figure 5a (horizontal and vertical separations at CPA are presented in separate graphs). The numbers attached to small boxes represent the corresponding encounters. Dashed lines present the range of ±0.5 Nm in the horizontal plane and ±500 ft in the vertical plane. They are introduced only for easy visual comparison. Horizontal distances obtained using the model are within the range of -0.24 Nm to +0.39 Nm relative to the real one, i.e. to the diagonal. The range of vertical distances is broader and relative to real life cases it goes from -773 ft, for encounter 5 where only TA is recorded in SDCPN model, up to +345 ft, for encounter 4 (Figure 5a). In encounter 2 which is at previous Level 3 recognized as the encounter with difference in sense, vertical separation at CPA is greater then in reality (1064 ft vs. 1000 ft). The corresponding range of values without encounter 5 is from -533 to +345 ft. Generally, vertical separation in two encounters (2 and 4) is greater then in reality, while for the rest it is lower. In the SDCPN model case, in all encounters with RA issued, except in encounter 1, the vertical separation at CPA is greater than 1000ft (vertical separation minima).

The comparison between InCAS model and Real Life data is represented on Figure 5b. Also in this case differences exist between the InCAS model and reality. The differences in horizontal separations range between -0.21 Nm and +0.44 Nm. Differences in vertical separations at CPA range between -719ft in encounter 5 were TA is issued up, and 585 ft in encounter 2. Generally, vertical separation in three cases (2, 4 and 7) is equal or greater then in reality, while in four cases (1, 3, 6 and 5 (case with TA issued)) it is lower. Having in mind that the InCAS model behaves similar as pilots in reality (meaning that strength value chosen for RA was determined from the recommended strength range), the possible reason for differences is related to differences in TCAS logic. In the case of the InCAS model, in all encounters with RA issued, except in encounter 1 and 3, the vertical separation at CPA is greater then 1000ft (vertical separation minima). Finally, mutual comparison between the ACAS SDCPN-based model and the InCAS model was performed.

The results are shown on Figure 5c. In the case of horizontal spacing the match is almost perfect, while differences in vertical spacing exist. They are in the range of -521ft (encounter 2) up to +204 ft (encounter 4). In encounters 1, 3 and 4 the corresponding vertical separations at CPA are greater for SDCPN then for the InCAS case. In the remaining encounters the vertical separation is smaller Additional analysis shows that in encounters 1, 3 and 5, the initial RA was not modified during the encounter while in encounter 2 and 6 it was modified in all cases (Reality, InCAS and SDCPN model). In reality in encounters 4 and 7, RA was not modified, while in the case of the InCAS and SDCPN models, RA was modified.

V. CONCLUSION

This research addressed systematic validation of a newlydeveloped mathematical model of Airborne Collision Avoidance System (ACAS). This newly-developed ACAS model has a significant advantage over models of similar kind when it comes to mathematical analysis and flexibility to integrate the model with other elements of an ATM operation.

The key question then is whether these advantages come at the cost of model validity. The investigation of this validation question has been addressed in this paper.

First in section II the conceptual ACAS model has been specified, including an explanation of the improvements made relative to the ACAS conceptual model in [START_REF] Netjasov | Stochastically and Dynamically Coloured Petri Net Model of ACAS Operations[END_REF]. Next, in Section III the model validation principles of [START_REF] Sargent | Verification and Validation of Simulation Models[END_REF] and [START_REF] Balci | Verification, Validation and Testing[END_REF] have been used to develop a systematic validation process using both real life data and an existing, well-tested ACAS simulation model. Subsequently this systematic validation process has initially been applied to the novel SDCPN-based ACAS model. The results obtained show that the systematic validation process works well, and that the novel ACAS mathematical model is working as well as the well-tested ACAS simulation model does. Some follow-up research activities are foreseen. At this moment the number of real life ACAS encounter data is rather limited. Hence much more real life data will be collected in order to complete the systematic validation. The advantage of working with a larger set of real life encounter data is that at each validation level we can make use of a statistical hypothesis testing formalism. Another research activity is to enhance the SDCPN-based model with the TCAS version 7.1 modifications.

The novel ACAS model can easily be integrated with other TOPAZ simulation models for various existing or future operational concepts. This way it can for example be assessed how well ASAS based approaches work together with ACAS, and what the total risk reduction is of a combined use of airborne safety nets.

Figure 3 .

 3 Figure3. The proposed Validation Process[START_REF] Netjasov | Risk Analysis and Safety Assessment of Air Traffic Control System: Model of Airborne Collision Avoidance System Operation for Safety Assessment of Air Traffic Control Operational Concepts[END_REF][START_REF] Netjasov | Validation of SDCPN Model for TCAS II Version 7 Operation in ATM[END_REF]

Figure 4 .

 4 Figure 4. Validation Approach

Figure 5 .

 5 Figure 5. Level 4 Validation Results

TABLE I

 I

	Start of Model Validation
	Level 1 Is TCAS activated?		no	Model Modification
		yes
		Level 2		no
	Are the same TCAS events
		occurring?
		yes
		Level 3		no
	If RA occurs, is resolution manoeuvre
		similar?
		yes
	Level 4 Are corresponding horizontal and vertical	no
	separations after implementation of issued
	RA at CPA close enough to the same
	separation in Control case?
		yes
	End of Model Validation
		.	MODELS CHARACTERISTICS [34]
		ACAS SDCPN model	InCAS
	Model Nature	Stochastic (encounter type model)	Deterministic (encounter type model)
	Purpose	Risk and Safety Assessment of TCAS operation	Analysis of ACAS encounters taken from real radar data
	TCAS II Logic	Model of TCAS II Logic	Real TCAS II logic, provided by MITRE Co.
				Step change
	Altitude change	Continuous	(quantization of 25ft and
				100 ft)
	Vertical speed,			In case of recorded radar
	Ground speed, Magnetic	Constant during encounter	data they are variable during encounter, otherwise
	heading			they are constant
		Included,
		with randomly delayed	Included,
		reaction	with delayed reaction and
		(without return to original	idealised pilot response
	Pilot reaction	trajectory or original	(with return to original
		vertical speed after Clear of	trajectory or original
		Conflict) and with	vertical speed after Clear of
		possibility to refuse to act	Conflict)
		according to issued RA
		Included as active
		(responsible for separation)
	ATCo role	or passive (when pilot is reacting according to RA).	Not included
		Reaction is randomly
		delayed.
	Reliability of technical system	Included (failure rates)	Not included

TABLE II

 II

	.	LEVEL 1 VALIDATION RESULTS (ACAS SDCPN MODEL VS.
			REAL LIFE ENCOUNTERS DATA)
			Real Life Encounter Data
			No	Yes
	SDCPN	No Yes	0 0	0 7

TABLE III

 III

	.	LEVEL 2 VALIDATION RESULTS (A DIFFERENCE HAS BEEN
		IDENTIFIED ONLY FOR ENCOUNTER 5)
		No evennt	TA	RA	RA	RA MAC
	a)		Real Life Encounter Data

TABLE IV

 IV

	.	LEVEL 3 VALIDATION RESULTS (A DIFFERENCE HAS BEEN
		IDENTIFIED ONLY FOR ENCOUNTER 2)
		RA		RA*		RAMAC
		u/d	d/u	u/d	d/u	u/d	d/u
	a)	Real Life Encounters Data
		u/d

ACKNOWLEDGEMENT This research was conducted with support from the "RESET -Reduced Separation Minima" project, commissioned by the European Commission Directorate General for Transport and Energy (DG TREN), under the FP6 Program (2006-2010), and project "Methods for evaluation of development scenarios of the air transport system in Serbia (airlines, airports, ATC) -safety, efficiency, economic and environmental aspects", commissioned by the Ministry of Science and Technological Development of the Republic of Serbia, under the Program for Research in Technology Development in the period 2008-2011.

For the validation of the developed model, authors are very grateful to Prof. Peter Form and Dr Jens Gottstein from the Institut für Eisenbahnwesen und Verkehrssicherung, TU Braunschweig, for providing real life TCAS encounters data, and to Mr. Garfield Dean from the EUROCONTROL Experimental Centre, Bretigny-sur-Orge, France, for providing InCAS Version 2.6, together with insightful explanations.

AUTHOR BIOGRAPHY