Mihaela Sighireanu

T his thesis is dedicated to the analysis of low-level software, like operating systems, by abstract interpretation. Analyzing OSes is a crucial issue to guarantee the safety of software systems since they are the layer immediately above the hardware and that all applicative tasks rely on them. For critical applications, we want to prove that the OS does not crash, and that it ensures the isolation of programs, so that an untrusted program cannot disrupt a trusted one.

The analysis of this kind of programs raises specific issues. This is because OSes must control hardware using instructions that are meaningless in ordinary programs. In addition, because hardware features are outside the scope of C, source code includes assembly blocks mixed with C code. These are the two main axes in this thesis: handling mixed C and assembly, and precise abstraction of instructions that are specific to lowlevel software. This work is motivated by the analysis of a case study emanating from an industrial partner, which required the implementation of proposed methods in the static analyzer Astrée.

The first part is about the formalization of a language mixing simplified models of C and assembly, from syntax to semantics. This specification is crucial to define what is legal and what is a bug, while taking into account the intricacy of interactions of C and assembly, in terms of data flow and control flow.

The second part is a short introduction to abstract interpretation focusing on what is useful thereafter.

The third part proposes an abstraction of the semantics of mixed C and assembly. This is actually a series of parametric abstractions handling each aspect of the semantics.

The fourth part is interested in the question of the abstraction of instructions specific to low-level software. Interest properties can easily be proven using ghost variables, but because of technical reasons, it is difficult to design a reduced product of abstract domains that allows a satisfactory handling of ghost variables. This part builds such a general framework with domains that allow us to solve our problem and many others.

The final part details properties to prove in order to guarantee isolation of programs that have not been treated since they raise many complicated questions. We also give some suggestions to improve the product of domains with ghost variables introduced in the previous part, in terms of features and performances. i

Chapter 1 Introduction S oftware systems are everywhere. But can we trust them? Indeed, some software are in charge of important missions and failures can be very grievous, such programs are called critical. There are famous examples with various kind of consequences, for instance:

-privacy: Heartbleed bug in OpenSSL (2014) * † ; -money: maiden flight of Ariane 5 launch vehicle (1996; $370 million for the payload only [START_REF] Dowson | The Ariane 5 software failure[END_REF]); -deaths and casualties: Toyota electronic throttle control system failure (2005; 89 deaths); surface-to-air (SAM) missile Patriot MIM-104 failure (1991; 28 deaths in its own army, not counting the deaths caused by the missile that the Patriot did not intercept) [START_REF]Patriot Missile Defense: Software Problem Led to System Failure at Dhahran, Saudi Arabia[END_REF]; Therac-25 radiation therapy machine (1985-1987; 6 accidents) [START_REF] Leveson | An investigation of the Therac-25 accidents[END_REF].

Even excluding such extreme cases whose cost cannot really be evaluated, NIST ‡ estimated in 2002 that software bugs cost U.S. economy between $22.2 and $59.9 billion annually [START_REF]The Economic Impacts of Inadequate Infrastructure for Software Testing[END_REF]; one may imagine how costly bugs are worldwide in 2020. Tests and documentation represent a vast majority of cost and efforts in critical software development, and still represent a decent part of most regular program development. Analyzing programs to detect errors is a crucial problem as it may avoid calamitous consequences critical software may have, and decrease the cost of software development while still producing more reliable applications.

Program analysis faces various challenges depending on the kind of software we study, the programming language in which it is written and the properties in which we are interested. These questions decide which methods can be used and how.

MOTIVATION

3 was written in a high-level language (Ada ¶ ), following very strict guidelines, and was extensively tested * * ; still Ariane 5 crashed. For this kind of software, we need stronger guarantees: we do not only need to detect probable bugs, but we want to prove that bugs are impossible. The definition of "bug" depends on the context: we may want to guarantee the absence of crash, the absence of private information leak or that the result is correct according to some specification. These properties are about the behavior of the program, they are semantic properties.

It is well known that interesting semantics questions cannot be solved using an algorithm, for a good definition of interesting: this is Rice's theorem. However, nonautomatic methods cannot be used on large codebases, or at the cost of a very long time of expert work. Even if automatic methods cannot be infallible, we can strive to make them successful on typical source codes. We may also develop semi-automatic methods that do most of the job automatically but can be manually helped (preferably, by a non-expert) when it fails.

Motivation

The main motivation of our work is to prove security properties on an embedded operating system (OS) from an industrial partner. This OS hosts several programs, some are provided by the manufacturer and trusted, other are installed by the customer and untrusted. In addition to the absence of run-time errors (RTEs) in the OS, which is simply a safety property, we want to prove more complex properties like the absence of information flow from a private source to a public memory, so that an untrusted program cannot access data of trusted applications. Likewise, we want to prove that public variables cannot interfere with trusted software, this way the behavior of trusted programs does not depend on the behavior of untrusted ones.

In general, these properties are difficult to prove, and even difficult to express: they cannot be formalized as sets of traces, but only as sets of sets of traces, what some people call hyperproperties † † . The absence of interference does not necessarily require that the untrusted program never writes in a trusted memory, but only that whatever it writes, the trusted program runs in the same way. We cannot directly use this property as we do not know the program running in our host platform. A first step is to say that the trusted memory can be modified by the untrusted program, but will eventually be restored its initial value before the trusted program reads it. Instead of comparing sets of traces, we only compare two traces: one real trace where the interaction happens, and one forged when the interaction is suppressed. If both traces match before the trusted program reads the memory, then, from its point of view, it is just like if the interaction never happened. This method can already be used in some parts of the OS where we need 4 CHAPTER 1. INTRODUCTION that a memory keeps its value at some times, but it is still too flexible to be used about user-level programs, as we would still need to know their source codes. The ultimate simplification is to simply forbid programs to read or write memory belonging to other processes. Though it seems very natural, it is in fact a strong over-approximation of the actually desired property. This property is interesting as it is a safety property, and that it is indeed enforced by memory management features of modern processors.

We still have to prove that the processor makes a correct usage of these protection features. They are way beyond the scope of C: these features cannot be controlled using pure C, as they are very dependent on the architecture, and properties about these features cannot be expressed in the memory model of C. We need to look at a lower level: we need to analyze assembly source code. In our case, we are using a processor with x86 architecture.

Of course, most OSes, and ours is not different, are not entirely written in assembly. The largest part is written in C, and only hardware-dependent pieces are written in assembly. This hides two problems. Firstly, we need to analyze mixed C and assembly source code, though they are very different languages: they have very different memory model, and while C is well-structured, assembly control flow is jump-based. The other problem is the kind of operations required by low-level software, which are unusual, or even illegal in standard C.

Since we want to prove properties on software, abstract interpretation is a convenient framework. It allows building sound analyses which are able to find all possible errors (at the risk of finding a bit more). The choice of the theoretical background was mainly guided by the application: a large part of this work is implemented in a development version of Astrée static analyzer. Astrée is a static analyzer relying on abstract interpretation developed since 2001 that managed to prove the absence of RTEs in large industrial embedded software [START_REF] Blanchet | A static analyzer for large safety-critical software[END_REF]Cou+01;[START_REF] Cousot | The Astrée Static Analyzer[END_REF]Cou+06a].

Because of the application, we not only need to find a solution to the problems we mentioned: this solution has to be efficient enough to be usable on real-world programs containing hundreds of thousand to millions lines of code.

Overview

We first consider the question of analyzing mixed C and assembly code. The first part is dedicated to a definition of the semantics of mixed C and assembly. Of course, we do not use C (which is very complex) or the whole assembly language, but a simpler model instead of C, and a fragment of x86 instructions. First, we introduced this Clike language from syntax to semantics. This language is not a subset of C as we need extra hypotheses that are undefined, unspecified or implementation-defined behaviors in C. We try to highlight these differences. Then, we introduce the memory model for assembly and two semantics. The first semantics is very concrete, it is very close to how computers work; the second semantics is slightly more symbolic to make interaction with C easier, but it forbids some unwanted behaviors. We explain how to write mixed C and assembly code, and give examples of complicated cases that justify why we cannot 1.3. OVERVIEW restrict the semantics too much. We explain the semantics of mixed code by looking at each kind of statement, but we do not give a full formal definition as it is very heavy and do not help to understand. The last chapter is about a library that we developed for Astrée, which is in charge of assembly-related problems, mainly deciding the length of assembly instructions to resolve dynamic jumps.

The second part gives some context needed for the following. We give a quick overview of abstract interpretation, and especially of products of domains. This does not intend to be exhaustive as a lecture. We also introduce Astrée, and explain some details about the implementation. These details are useful to understand the implementation choices made in the following.

The third part is about the abstraction of mixed C and assembly code. We divide the work in instruction categories. For each category, we present our abstractions, and the changes they imply in Astrée.

We mentioned that low-level source codes perform strange operations. This problem is discussed in the fourth part. Such cases can be solved by adding ghost variables: they are quantities that does not directly appear in the program but help to represent states. We found analogous notions in other domains; for instance, in physics, the law of conservation of energy can be used to solve many problems, but requires the total energy of a system to be explicit, which is an artificial value that has no existence in real life, just like the limit of the system is arbitrary and human-made. Likewise, given a problem, any solution that uses change of variables can be rephrased with ghost variables, e.g. variable change in an integral computation or coordinate rewriting in a geometric problem. In static analysis, ghost variables are very useful as well, but they are difficult to use in a decentralized way with multiple abstract domains. In the fourth part, we build a product of abstract domains that makes domains able to share ghost variables, and manage them in a fully distributed way, while allowing domains to cooperate to improve their abstract values. This allows the composite domain to express more complex properties.

The fifth part gives some perspectives and insights. We propose many extensions and improvement of the work presented in previous parts. In particular, we propose some advanced properties to prove that require the support of assembly. We also mention many optimizations of the product of domains discussed in the fourth part, and some extensions to allow even more complex domains and to make the product more general, so that it can be used in many contexts.

Chapter 2

The x86 Architecture and System Programming T his chapter explains relevant parts (for the following chapters) of the x86 architec- ture and how these low-level features are used to build an OS. These explanations can be arranged in two categories. The first kind is about the actual features of the processor. Most of this information might be found in the Intel Architectures Software Developer Manual [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF]. The second kind is the set of methods and practices employed to build an OS using these features. In this matter, there is no absolute truth but widespread good techniques and traditions. The Intel Architectures Software Developer Manuals gives some pieces of advice but the main part of these techniques rather comes from tutorials and documentations about existing OSes, since most of them work in similar ways. A most useful resource in this matter is OSDev [START_REF] Osdev | [END_REF].

An Overview of the x86 Architecture

A Piece of History

The x86 architecture was born in 1978 with the release of the 8086 microprocessor, the first Intel's 16-bit processor. As x86 maintained backward compatibility, novelties introduced in the 8086 processor are still part of the modern x86 processors. Yet, the 8086 lacks fundamental features that are essential today, for instance to handle multitasking. Consequently, since the 8086, the x86 architecture was enriched to deal with new needs.

A first important landmark is the introduction of a new memory management mode by the 80286 processor in 1982. In the 8086 framework, the physical memory was addressed directly by the program (real-address mode), preventing to run two programs simultaneously as they could interfere. The 80286 added the protected mode: in this framework, each program can only address its own virtual view (called segment) of the memory, and the processor takes care to map views on separate areas of physical memory as instructed by the system. Its successors (and contemporary) until the release of the Pentium were named on the pattern 80x86, giving by apheresis the name x86.

CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING

The 80386, released in 1985, was the first 32-bit x86 processor. It is also named i386, hence the informal name of the 32-bit generation of x86 architecture, otherwise called IA-32. The 80386 brings a lot of changes. To offer better support for 8086 programs, a new virtual mode was created to run real-address mode applications in an isolated environment in protected mode. These programs were not designed to work with segmentation or paging, and at this time, it was an important issue to be able to still run these old programs on new processors. As such old software was gradually deprecated and replaced by new programs designed to work with modern memory protection features, virtual mode became less and less useful. The change to 32-bit generation did not only extend registers, but also the external bus, allowing to address a much larger memory (from 16 Mio to 4 Gio, see section A.1 "Architecture" (page 369) for units). In parallel, a new memory management feature was added: paging. It provides a much more flexible control of the memory by allowing to map segments to non-contiguous small blocks of memory. With time, paging became more and more powerful so that it entirely subsumed segmentation, which became unused and eventually removed from the 64-bit generation (when operating in 64-bit mode).

Due to backward compatibility, even the very last x86 processor starts just as the 8086. The system is in charge to gradually change execution mode to the desired one. During the following description of the x86 architecture, some points may seem odd, even wrong. One should keep in mind that this is the result of many layers to add new features while keeping the old ones.

Characteristics of the 32-bit x86 Architecture

An architecture (or instruction set architecture) is an abstraction of a processor: it describes the available instructions, their semantics and their encoding.

The processor we are interested in is the Pentium MMX. Since it is a 32-bit, we will not consider 64-bit architecture. However, we cannot escape the influence of 16-bit processors, because of backward compatibility.

The x86 architecture is a CISC (complex instruction set computer) architecture, i.e. there is a lot of instructions: there are multiple versions of simple operations (depending on the type of the operand: literal value, register or memory operand) and very specialized instructions. In particular, most instructions can read or assign registers as well as memory. Once encoded into machine language (binary encoding), each instruction will take between 1 and 15 octets (tight bounds).

In x86, each byte is 8-bit long, that is a byte is an octet. This architecture uses little-endian byte order and allows any unaligned access (though they might be slower). All integer operations assume two's complement representation.

In the following, we will only focus on the Pentium MMX. In fact, we will not look at any feature that makes the Pentium MMX different from the P5, or most 32-bit x86 processors. However, really ancient processors may not have all the thereafter explained features, and more recent x86 processors may have more (while keeping compatibility).

AN OVERVIEW OF THE X86 ARCHITECTURE

Conventions

Most notations follow the Intel's manual [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF]. This way, it is easy to find complementary information. Moreover, due to Intel's influence, these notations are quite common in a lot of resources. Let us explain them.

The first point is notation of integers in non-decimal form. Hexadecimal numbers have the H suffix and use numbers 0 to 9 and A to F. Binary numbers have the B suffix and use numbers 0 and 1. For instance 42 = 2AH = 101010B. In most programming languages 2AH is written 0x2a.

When drawing data structures, lower addresses are at the bottom and the byte-offset is written on the right. Bits are numbered from right to left. For instance, the array of the 16 hexadecimal 4-bit digits in increasing order can be equivalently written as on Figure 2.1 or Figure 2.2. The number of bytes per line depends on the context. Usually, there are 1, 2 or 4 bytes per line.

2: Hexadecimal numbers by 32-bit packs

There is also a notation to extract some bits: a[c:b]. This means "bits b to c of a". For instance if a is 12ABCDEFH, a [27:4] = 2ABCDEH (see section A.1 "Architecture" (page 369) for a more detailed explanation). Bounds are reversed to match the bit order of figures. Outside the context of figures, natural order is preferred (first index before last index).

Reserved parts are in gray. A constant written in such a cell means that this field shall always contain this constant. Otherwise, the content may be arbitrary.

When speaking about bits, saying it is cleared means that its value is 0, and we say it is set if its value is 1.

CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING

Execution Environment

Registers

Among other resources given to program running on the processors, there are the registers. They are small memories that exist only in the processor: they are not mapped in memory, and consequently they have no address, thus no pointer can point to them. Let us remark that "a memory" designates anything that can store data, but "the memory" designates more precisely the memory external to the processor accessed by dereferencing, typically the random-access memory (RAM).

There are several categories of registers; we are going to present them in this subsection.

General-Purpose Registers

Some registers are available for the user without restriction. They are called generalpurpose registers (GPRs), or sometimes GP registers. There are 8 GPRs, all of them are 32-bit wide. They are named EAX, ECX, EDX, EBX, EBP, ESP, ESI, and EDI. As one may have noticed, EBX come after EDX, especially when it comes to encode the instructions. This usually has no importance, but sometimes, it explains surprising facts.

In addition to these registers, other names are defined to designate parts of these registers. For EAX, the 16 lower bits are called AX, the 8 lower bits form AL and the bits 8 to 15 are AH. This is illustrated on Figure 2.3. Similar names are given to analogous parts of ECX, EDX and EBX. The 16 lower bits of EBP, ESP, ESI and EDI are respectively called BP, SP, SI, and DI. The "E" prefix stands for "extended", since 32-bit registers are extension of the former 16-bit registers.

These registers can be freely used to store any kind of data (integer or pointer, since there is no typing at this level), without restriction, especially whatever the current privilege level, and without any kind of check. While it is possible to store anything, it is not always wise. Indeed, some instructions favor some registers. For instance, addition is more efficient when using AL, AX or EAX: it is both faster and the encoding is smaller. Moreover, some instructions exist only for some registers. For example, string operations can only be performed using ESI as a pointer to the source operand and EDI as a pointer to the destination operand, hence their names (see below). Likewise, when performing a stack operation, ESP will be interpreted as a pointer to the top of the stack. Moreover, while all GPRs may be dereferenced to be used as memory operand, they are not understood in the same way: during a dereference, pointers must be translated to physical memory address, but this translation is done slightly differently depending on the register. This will be explained with more detailed later (see subsection 2.3.2 "Segmentation" (page 18)).

Even if there is no intrinsic constraints on the respective usage of GPRs, their names carry some hints:

-ESP: Stack Pointer -EBP: Base Pointer (in stack) -ESI: Source Index (for string operations) -EDI: Destination Index (for string operations) -EAX: Accumulator (for arithmetic operations) -ECX: Counter (for string operations) -EDX: Data pointer (for I/O operations) -EBX: Base pointer (for 16-bit-style addressing) While the 4 former are credible, the others look more dubious, or even apocryphal. In practice, ESP and EBP are indeed bound to stack management.

It is important to remember that values are not typed in x86: they are just binary sequences. It is true everywhere, but especially important to emphasize about GPRs since they have no special meaning. The content of a register, or a memory operand, can be interpreted as a pointer, a signed integer or an unsigned integer, depending on the instruction. That's why there are both MUL and IMUL: the former performs an unsigned multiplication, while the latter performs a signed multiplication. Thanks to two's complement representation, some computations (such as addition) are the same in both cases.

EFLAGS Register

The EFLAGS Register is another very important 32-bit register even for non-system programs. It keeps 3 kinds of flags: status, control and system flags. Status flags give some information on the last executed instruction. They are automatically updated and consequently, from the software point of view, they are mainly intended to be read. A control flag is rather meant to be modified: its value modifies the behavior of some instructions. System flags are dedicated to system management and should not be modified by an application program. Moreover, as a protection feature, one needs the highest privileges to change these flags. The layout of the EFLAGS register is shown on Figure 2.4. Some bits (in gray on the figure) cannot be changed.

The EFLAGS register cannot appear directly as an operand. To read or write it, one can either use dedicated instructions that set or clear specific flags, or copy the whole register from or to the stack. In most cases, flags are written and read automatically without manipulating the EFLAGS register.

There are 6 status flags: Overflow Flags (OF), Sign Flag (SF), Zero Flag (ZF), Auxiliary Carry Flag (AF), Parity Flag (PF), and Carry Flag (CF) 0 0 0 0 0 0 0 0 0 0

I D V I P V I F A C V M R F 0 R F I O P L O F D F I F T F S F Z F 0 A F 0 P F 1 C F Figure 2
.4: EFLAGS register arithmetic instructions (such as addition, multiplication. . .) depending on the result. For instance, the Zero Flag is set if the result is null, and cleared otherwise. Testing status flags is the main way to execute code conditionally, excluding tricks like dynamic jumps. Conditional jump instructions might simply test the value of a flag or a combination of flags. Moreover, most of these instructions have aliases. For instance JZ (jump if zero) tests if ZF = 1, just as JE (jump if equal) which is a mere alias for convenience. Indeed, comparison instruction (CMP) sets status flags just as SUB and does not modify the operands: ZF is set if the difference is zero, i.e. if the operands are equal. Even if it is equivalent, it is more natural to test whether the result of SUB is zero and whether operands of CMP are equal; that why JZ and JE are aliases. Complex conditions are used to provide convenient comparisons. For instance JA (jump if above, the unsigned strict comparison) tests if ZF = 0 and CF = 0 without the need to use two instructions. Since these registers are automatically written by arithmetic operations and read by conditional operations (including, but not limited to, jumps), it is very unusual to read or write status flags manually and x86 does not make it easy.

There is only one control flag: Direction Flag (DF). It is used to change the behavior of string operations (MOVS, CMPS, SCAS, LODS, and STOS). String operations process contiguous block of data, by applying an instruction to each value in this block. DF flag sets the direction in which the string is processed: from high to low addresses, or the opposite. This flag can be set or cleared using dedicated instructions. We will not take care of this flag anymore since we are not interested in string operations.

The other flags are system flags. One needs the highest privilege to modify them, and they control some system features or indicate capabilities of the processor. Most are not relevant for the following, the others will be detailed in due time.

Segment Registers

Segment registers allow some control on logical-to-physical address translation. There are 6 of them (CS, DS, SS, ES, FS, and GS) and are 16-bit wide. Their role is explained in subsection 2.3.2 "Segmentation" (page 18).

Instruction Pointer

The instruction pointer (EIP), as suggested by its name, points to the next instruction to run. It is impossible for software to access it directly: it is automatically controlled by control-flow instructions (jumps, calls. . .). Though, one can take advantage of these instructions to get the current value or to assign an arbitrary value into EIP, it goes without saying that this is a very poor idea in most application programs. But sometimes it can be necessary for system to use such trick.

Control Registers and Other System Registers

While all aforementioned registers may be manipulated (at least partially) by an application program, there are others families of registers that are only intended to be used by the system. The highest privilege is required to access them,and they are used to control the mode of operation, memory management features and other sensitive parameters. Relevant registers, with respect to our scope, will be detailed at an opportune moment (especially subsection 2.3.2 "Segmentation" (page 18) and subsection 2.3.3 "Paging" (page 25)). Important such registers are the control registers, named from CR0 to CR4.

Stack

Most languages need a stack to store information about active subroutines. The stack is not a distinct hardware feature, instead it takes place in the memory as a set of consecutive cells. Nevertheless, x86 provides several operations to easily operate on a stack. In x86, stacks grow downward, thus a PUSH will decrease the address of the top of the stack. For all the stack operations, it is assumed that ESP points to the top of the stack, that is the element with the lowest address of the stack. In addition to store or retrieve data from the stack, these instructions update ESP accordingly. Some relatively high-level instructions handle entire call frames, managing both EBP and ESP to surround them, and keeping appropriate copies on the stack to be able to restore the previous state. Thus, even if the stack is a purely logical structure, x86 processors include several instructions to handle it pleasantly.

Privileges

Privileges have already been mentioned briefly. This is a mechanism to restrict some sensitive operations only to some trusted programs (especially, the system) and to protect these programs against less privileged tasks.

There are 4 privilege levels, called rings, numbered from 0 to 3. Ring 0 is the most privileged level, while ring 3 is the least * . A numerically higher is in fact a lower privilege. Most tasks (in particular, application programs) run in ring 3, only the OS kernel should run in ring 0. Rings 1 and 2 used to be the privileges of OS services, such as drivers. Nowadays, they are much less used for reasons related to hardware and software: on the processor side, the 64-bit generation of x86 mostly dropped segmentation in favor of paging, which has only a protection model with 2 levels; from the OS side, limiting * Hence the name "ring": privilege levels are like a succession of included circles, where ring 3 is the outermost and ring 0 the innermost. This diagram can be found in many places, especially [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF]. This is the traditional representation, though it is counterintuitive from a set-theoretic point of view (since the set of allowed operation for ring 3 is included in the set of allowed operation for ring 0). We may rather interpret this representation like battlements in a fortified city: ring 0 is the innermost and thus the most secured place.

CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING

hardware-dependent code is an important issue for portability and maintainability, yet some architectures only provide two privilege levels, thus on x86, two rings are left unused.

Sometimes, it is useful to temporarily change ring to allow an application program to perform privileged operation, such as allocating memory. Such calls are known as system calls. There are several ways to increase privilege safely: we do not want the unprivileged task to run arbitrary code with high privilege. These transfer mechanisms are detailed in section 2.4 "Calls, Interrupts and Exceptions" (page 28) and section 2.6 "System Calls" (page 36). Whatever the way by which the transfer happens, it is crucial to prevent the system to access sensitive data, or to run protected code, on behalf of an application program.

To achieve delegation to higher privilege code while ensuring protection, x86 uses three kinds of privileges:

-The current privilege level (CPL) is the privilege of the currently executing task.

-The descriptor privilege level (DPL) applies for data segments and gates (safe proxies for privileged functions see subsubsection 2.4.2.3 "Inter-Privilege Calls" (page 31)). They describe the required privilege to be allowed to access the data (in the case of a data segment) or to run the code (for gates).

-The requested privilege level (RPL) is an override to access a data segment or a gate with a numerically higher privilege level (less privileged). This is useful to execute code or access data on behalf of a less privileged task.

Modes of Operation

Processor of the x86 family can run in three modes. Each mode determines a set of available features and instructions. The legacy mode is called real-address mode. This is the only mode of the 8086 thus, for compatibility reasons, this is the mode of the processor at startup. Most protection features are unavailable, so it is strongly advised that non-legacy software does not use this mode.

The normal mode of operation is the protected mode. It can be accessed from real-address mode and provides most features and instructions, in particular memory protection features (hence the name) needed for multitasking. Moreover, since it is designed to be the standard mode of operation, this is also the most optimized mode. In most cases, the first task of the system is to transit from real-address mode to protected mode. Tasks running in protected mode can be run with a special parameter so that the processor emulates the real-address mode in a protected framework, allowing running software designed for the 8086. While not technically being a distinct mode, this feature is called the virtual-8086 mode.

The third mode is the system management mode. This mode is intended to run code to manage hardware-specific features such as power management. It provides an easy way to execute this code transparently (OS or application programs are automatically suspended and resumed) in a isolated environment. This mode is out of our scope.

Memory Protection Overview

Real-address mode allows programs to access physical memory directly. The only way to run multiple tasks in such a mode would be to use disjoint addresses. This is not feasible in practice: when choosing the addresses to store global variables and functions (during compilation, typically), one would have to know which are available. In addition to the annoyance due to accidental interference of genuine programs, there is no way to protect data from an adversarial program.

To tackle this issue, protected mode provides features to isolate memory space of programs. There are two consecutive mechanisms of address translation: segmentation and paging. Enabling paging requires segmentation to be currently active. In normal operation, both segmentation and paging are enabled. The state where segmentation is enabled but not paging only appear shortly during boot. Since there are two translation steps, there are three address spaces: logical, linear and physical addresses. Segmentation translates logical addresses (also known as far pointers) to linear addresses. A logical address is the address from the program point of view. Such an address has no meaning by itself, it should be interpreted in a segment. A segment is a contiguous memory zone characterized by a base address (the beginning of the segment), a size, and some flags. Segments are usually defined globally, but task-specific segments CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING are also available. A logical address comes with a segment selector (stored in a segment register), which is an index that specifies in which segment the logical address must be interpreted. Then, the logical address is added to the base address of the segment yielding the linear address. Checks are performed to ensure that the linear address does not exceed the segment limit. This process is shown on the left part of Figure 2.5.

Depending on whether paging is enabled, the linear address may undergo another translation. If paging is disabled linear address is directly interpreted as a physical address. In this case, to ensure memory isolation, tasks must use disjoint segments. This lacks flexibility since it implies that each task requires a contiguous block of memory. So, memory allocation may require entire segments to be shifted if there is no space left after the end of the segment. In addition, holes cannot be used easily and segmentation does not allow easily storing allocated but unused memory on an external device (typically a hard drive or a SSD) to make room for more frequently used data (this technique is known as swapping).

To overcome these issues, we usually use paging which is a second translation step that provides a useful mapping from linear addresses to physical addresses. Linear address space is cut into small pieces called pages which are mapped anywhere in memory. So, the processor determines to which page the linear address belongs and the offset inside this page. Then it finds the physical base address of this page in a dedicated taskdependent structure and adds the offset to get the physical address. In this framework the granularity of memory is coarser: we cannot allocate less than a page. Yet, pages are small (usually 4 Kio), so this is not an issue. This translation is illustrated on the right part of Figure 2. 5.

Paging solves all aforementioned issues. To allocate memory, the OS just needs to find available space to put the pages. They do not even need to be adjacent. Swapping can be done at page level: when the processor finds out that the requested page is not in memory, it asks the OS to move it into memory before proceeding. To achieve memory isolation, pages must not overlap, but segments are not important anymore: the same linear address space is translated differently in the context of different tasks. Consequently, in most OSes, segments are shared across all tasks and span over most of the linear address space, so that the segmentation is almost trivial. This led to the removal of most capabilities of segmentation in the long mode (64-bit mode) of x86_64.

Both segmentation and paging will be explained with more detailed in section 2.3 "Memory Management" (page 17).

Assembly Language

To be understood by the processor, instructions are encoded into binary sequences, this is machine code. Yet, this format is clearly not human-friendly. Instead, we use an assembly language (or assembler language): it is an understandable language very close to the machine code.

There are several syntaxes for x86 assembly, the most popular ones are Intel's and AT&T's. We will use the Intel's syntax in the following since it has slightly more in common with other languages (direction of assignments) and has more understandable syntax for complex addresses. Moreover, it is closer to the Intel's manual [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF].

Assembly language use mnemonics followed by operands (separated by commas) to specify the instruction. A mnemonic is a keyword encoding the kind of instruction. For instance PUSH EAX is the instruction that pushes the content of EAX on the stack. The mnemonic is PUSH and EAX is the argument. An example with a binary mnemonic: ADD EAX, 5 adds 5 in the register EAX. More generally, when applicable, the source operand is on the right while the destination operand is on the left, just like in most languages. We can also remark that usually, mnemonics and other keywords (like register names) are not case-sensitive. Indirection is denoted with brackets. For instance, PUSH [EAX] pushes on the stack the content of the memory whose address is the content of EAX.

Assembly languages usually include labels. They are identifiers followed by a colon (e.g. my_label:) that marks a program point, making it easy to reference, for instance, as the target of a jump or call instruction. Labels do not exist in machine language, and they are replaced by the offset between the usage and the definition.

Assembly language is translated into machine language by an assembler program (or simply assembler). Fancy assemblers can provide more advanced features like macros or high-level style branching. This separates even more assembly language from machine code, so we will not use such features.

Memory Management

Address and Operand Sizes

The processor can use 16-bit or 32-bit addresses and operands depending on the current mode and configuration.

When using 16-bit addresses, the maximum legal value of a linear address (known as limit) or segment offset is 2 16 -1 (i.e. 64 Kio) and with 32-bit addresses, the limit is 2 32 -1 (i.e. 4 Gio). It is equally straightforward for operand size: 32-bit operand size means that the size of operand is 32 bits by default.

In real-address mode, the default address and operand size is 16 bits. In protected mode, it depends on a flag in the segment configuration. However, both address size and operand size can be overridden temporarily by adding appropriate prefixes to the instruction, so that an instruction in protected mode can access the whole memory (32bit address size), yet use AX register (16-bit operand size) by using the operand-size override prefix while in 32-bit mode.

The recommended setting is to use 32-bit sizes. This offers more capabilities (wider ranges for integers and larger addressable memory) while achieving better performances. Only serious constraints should impose 16-bit addresses and operands. But, we cannot totally escape 16-bit sizes because the processor starts in real-address mode, thus we are compelled to use 16-bit code at least until we reach protected mode.

Segmentation

While sharing the name, some terminology and segment registers, real-address mode segmentation and protected mode segmentation are very different. The similarity is that in both kinds, segments are contiguous zones of memory (like a segment in the mathematical meaning) and the logical address is used as an offset inside this segment. But the capabilities and the way to specify a segment have very few in common.

There are 6 16-bit segment registers: CS, SS, DS, ES, FS, and GS. While the way they specify the segment varies between real-address mode and protected mode, they are used in the same circumstances:

-CS (Code Segment): where instructions are fetched; -SS (Stack Segment): when dereferencing EBP or ESP, or performing a stack operation like a push or pop; -ES (Extra Segment): destination of string operations; -DS (Data Segment): for everything else. For memory operands, the automatic (register-based) selection of segment may be overridden, e.g. MOV EBX, [EAX] dereferences EAX in the data segment (DS) and stores the result in EBX, but MOV EBX, SS:[EAX] will look in the stack segment. So MOV EBX, [EAX] is equivalent to MOV EBX, DS: [EAX]. FS and GS segments are only used through such explicit overrides, as they are never automatically selected.

Real-Address Mode Segmentation

In 16-bit processors, registers are 16-bit long. This provides an addressable space of 2 16 octets (64 Kio). It rapidly became too small regarding the development of software.

To extend the addressable space, the Intel 8086 has a 20-bit memory bus, allowing it to use 1 Mio. To achieve 20-bit addresses with only 16-bit registers, the physical address is 10H × S + O (see Figure 2.6) where S is the content of the segment register (known as segment selector), O is the offset (logical address) and H is a suffix to denote that the value is denoted in hexadecimal (so that 10H = 16). Segments are 64 Kio long but spaced out by only 16 octets, thus they overlap. Consequently, the same linear address can be obtained with up to 2 12 distinct pairs of segment selector/offset. By using disjoint segments for code, data, and stack, we can avoid catastrophic mistakes like writing in the code, or executing data. But this is only a protection against some genuine mistakes, but it is useless against adversarial attackers, since the program can change current segments (as there is no privilege mechanism in real-address mode), and indeed execute data or write in the code. Real-mode is poorly suited to multitasking (executing simultaneously two programs requires them to statically use disjoint memory zones), so usually, there is only one program at a time in this mode, thus there is nothing to attack except the attacker itself. But in a multitasking mode, we need better memory protection.

Protected Mode Segmentation

This segmentation is much more flexible: segments may have arbitrary base address and length. Available segments are described by segment descriptors, which can be stored in a system-wide structure, the global descriptor table (GDT), or in a task-specific structure, the local descriptor table (LDT). Segment registers point to descriptors, specifying indirectly the segment.

Segment Descriptor

A segment descriptor is a 64-bit structure that specifies segments parameters. The layout is showed on Figure 2.7. The base address is the lowest linear address of the segment. The limit field specifies indirectly the size of the segment depending on the G (granularity) flag:

-if G is cleared, the limit is directly the 20-bit size, allowing sizes from 1 o to 1 Mio, by steps of 1 o; -if G is set, the size of the segment is 2 12 times the value of the limit field, allowing sizes from 4 Kio to 4 Gio, by steps of 4 Kio. The granularity flag allows favoring either the precision or the amplitude. In practice, we usually do not need to specify the segment size by less than 4 Kio since it is the smallest possible size for a page. Moreover, we usually need to specify large segments. So, the granularity flag is set in most use cases.

In a segment, the least logical address is 0 and the greatest is the limit (except for expand-down segments). The base is thus an offset added to the logical address to get the linear address. At this point, the least linear address is the base, and the greatest is base + limit.

The address translation of segmentation is rather trivial: it is a translation (in the mathematical meaning) with bound checks. But segment descriptors also include several flags to adapt segment settings and access rights:

-The DPL field specifies the privilege level of the segment, from 0 (most privileged) to 3 (the less privileged). For a code segment, it specifies the maximum privilege a task can have when running this code. -The P (present) flag specifies whether the segment is available for use. If not, most fields are ignored and the segment descriptor cannot be used.

CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING

-The AVL (available) flag is left available to the system developer.

-The D/B flag specifies the default address and operand size.

-Bit 21 of the second double word is reserved and should be 0.

-The S (system) flag specifies whether the segment is a system segment, or a code or data segment. System segments are special structures that will be explained later. -The Type field defines the type of segment (code or data) and access policy: execute-only or execute/read for code segments, or read-only or read/write for data segments. This field stores more information that will be explained later. When S bit is set, the segment descriptor is not a system segment, oddly. Let us describe the type field in the case of a non system segment:

-Bit 11 of the second double word is 0: Data segment: -Bit 10: expand-down. In this case, the greatest logical address is FFFFH (2 16 -1) if the B flag is cleared, FFFFFFFFH (2 32 -1) otherwise, and the least logical address is limit + 1. The base is still the least linear address of the segment. -Bit 9: write. If this bit is cleared, the segment is read-only, otherwise, write is allowed. -Bit 11 of the second double word is 1: Code segment:

-Bit 10: conforming. Whether code in this segment can be accessed by a less privileged task, and run with current privilege. -Bit 9: read. If cleared, this segment is execute only. Otherwise, read is allowed. -Bit 8: accessed. This bit is set when a selector to this segment is loaded, and should be cleared explicitly. It can be used for debugging or memory management, but is rarely useful, thanks to paging. We can remark that a segment may not be executable and writable at the same time. This is an obvious precaution to avoid catastrophic failures. But we can also note that the same linear address might be in multiple segments, thus it is still possible to have a piece of memory (in the linear address space) that is both executable and writable by reaching it through different segments. This can be avoided using paging (see subsection 2.3.3 "Paging" (page 25)) to avoid execution of data or dynamically modification of the code, but it can also be useful, precisely when dynamic code is needed.

Expand-down segments may seem weird, but they used to serve a reasonable purpose: they were designed for stacks since in x86, they grow downward. If one uses an expanddown segment for a stack, decreasing the limit of the segment allows more space to be dynamically allocated on the stack, by decreasing the first illegal address. Modern (even for our scope) virtual memory management does not use dynamic segments: allocation is performed at page-level.

Conforming segment is a concept used to share pieces of system's code that does not require high privilege to run. For instance, a math or cryptographic library may be useful in the OS, and can be made available to application programs since it does not involve privileged operations.

It is important to mention that the field Base is a 32-bit address scattered all across the segment descriptor, as shown by Figure 2.7. The limit is also cut in two parts. Building a segment descriptor is usually done by using bitwise operations, like in Listing 2.1. Though building a segment descriptor requires such obscure manipulations, the inverse process is transparent: the correct information is rebuilt by the processor automatically.

Segmenting the Linear Address Space

Segments can alias each other in arbitrary ways. When they do, a write in a segment may be visible in another segment, but at a different logical address, which is not advised! There are some standard ways to use segments for different use cases, and having a predictable behavior.

The conceptually simplest safe way to avoid problematic behavior is to use non aliased segments, i.e. segments that do not overlap. This is known as the multi-segment CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING model: each task use its own segments. This method ensures memory isolation at linear address space-level. But this model is really heavy to implement and isolation at linear address spacelevel is not always desirable. For instance, the DS segment is used for most memory accesses while the ES segment is used for the output of string operations, but we usually want to be in the same space: if several string operations are needed, the output of an instruction may be the input of the following. So, it would be nice for DS and ES to be the same segment. Another problematic case is the problem of dereferencing C pointer. Let us consider the example of Listing 2.2. The variable a is a global variable, it is stored in the heap, which is classically in the DS segment. On the other hand b is local, thus allocated on the stack (SS segment). In the function f, one only see x as a pointer, without knowing in which segment it should be interpreted, and, as shown by the example, it can be one and the other. To make the compilation feasible without having to transmit more information, the simplest way is to have the DS and SS segments to be identical. One could also craft some imaginative example involving function pointers to argue that CS and DS should be identical. More generally, we often want all segments to be identical parts of linear space. We do not lose so much in protection since segmentation has a quite austere style of protection while paging is much more precise. Based on this observation, a few variations exist. The most basic segmentation model is to have two segments (code and data) mapped to the entire linear address space. A variant is to have such a pair of segments for ring 0 and a pair of slightly shorter segments for user programs, but still starting at linear address 0. Segments are not identical, but where they overlap, each linear address corresponds to the same logical address in all segments. Some space at the end is unavailable using user-level segments, and thus can be used to store system data and code. The point of this style of segmentation will be detailed further in subsubsection 2.3.3.4 "Translation Lookaside Buffers" (page 28).

Global Descriptor Table

Until now, we have described the structure and the semantics of segment descriptors, but not how we can make them available to use. Indeed, they should occupy a privileged position to be used by the memory translation facilities of the processor. For that, segment descriptors are gathered in a structure called the global descriptor table (GDT).

The GDT is simply an array of segment descriptors. They are just put next to each other, with no gap. Thus, the byte-index of a segment descriptor is multiple of 8 (the size of a segment descriptor).

This structure is referenced by the GDTR register. It is a special register that contains the 32-bit address of the GDT in the linear address space, and a 16-bit limit, which is the offset (with respect to the base address) of the last valid byte of the GDT. Since all the fields are 8-byte wide, the limit should be the predecessor of a multiple of 8.

The GDTR is not directly accessible. It can only be written and read through special instructions, respectively LGDT (load GDT) and SGDT (store GDT). These instructions use a 48-bit memory operand: the 16 lower bits correspond to the limit part of the GDTR, and the 32 upper bits are the base address.

LGDT simply copies these bits from memory to register and SGDT copies in the other direction. These operations require the highest privilege.

The GDT is global to the whole system, and is designed to be long-lived. There is another structure that provides more dynamic segmentation: the local descriptor table (LDT). The most notable difference with a GDT is that the processor automatically updates the current LDT when switching tasks. Once again, because of paging, this feature is rarely used and, anyway, is out of our scope. Using LDT would not make so much difference in the following. Making the segments globally visible is not enough: we should have a way to reference them. This is the role of segment selectors. They are 16-bit identifiers to specify a segment descriptor that lies in the GDT (or LDT). Since segment descriptors are 8-byte long, their byte-indices are multiple of 8, thus, the last 3 bits are always 0. So, in the segment selector, these bits fulfill other purposes. The structure of a segment selector is shown on Figure 2.8. The 3 lower bits of the index are omitted, since always null. The TI (table indicator) bit specifies whether the segment is in GDT (if cleared) or LDT (if set). The RPL field specifies the privilege level at which the segment should be viewed. This matter has already been briefly discussed in subsection 2.2.3 "Privileges" (page 13).

Segment Selectors

CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING

The RPL allows accessing a segment as another task would do it, so that, a privileged task running code on the behalf on a less privileged task will not access sensitive code or data. For an access to succeed, the RPL must not be numerically lower (more privileged) than the CPL, and the RPL must not be numerically higher (less privileged) than the DPL of the requested segment.

It is worth noticing that the CPL of a task is in fact the RPL of the currently executing code segment.

To use a segment selector, it should be loaded into a segment register usually using MOV or POP, for non CS, and with control flow operations (CALL, RET, JMP. . .), for CS. Assigning a segment register is not an ordinary write. Indeed, segment registers contain a visible and a hidden part. The visible part is the segment selector, while the hidden part caches information of the segment descriptor. When a segment register is loaded, the hidden part is loaded as well, so that modifying a segment descriptor without reloading the segment register has no effect. The hidden part is managed only by the processor, software has no access to this part.

Privilege checks also happen when the segment register is loaded.

System Descriptors

Since the GDT is a global structure, it is interesting to make it store other kind of data than segment descriptors. Such structures are called system descriptors and have roughly the same layout. The types of system descriptors are:

-LDT segment descriptor. This was already mentioned and will not be detailed further. -Task-state segment (TSS) descriptor. This is a descriptor to a special structure, called TSS, that holds information about a task. This is used to save the current state of a task during context-switch. -Task-gate descriptor, a protected selector to a TSS descriptor.

-Call-gate descriptor, a protected pointer to code that allow inter-privilege control transfer. -Interrupt-gate and trap-gate descriptors. These descriptors contain pointers to procedures to run when the processor get an interrupt or an exception (see subsection 2.4.3 "Interrupts and Exceptions" (page 32)). These kinds of descriptors are not useful in GDT but are placed in interrupt descriptor table (IDT).

They are a very heterogeneous family of descriptors that fulfill various goals. The characteristic point they share is that they are useful only when they are globally accessible, from any context. Moreover, they all can be put in the GDT because they all have the same size (8 B) and the "type" field at the same place which allows the processor to identify their kinds.

Paging

Translation Mechanism

Paging takes place after segmentation and is optionally enabled, but requires segmentation to be enabled before. This is the modern way of managing memory. This mechanism maps the linear address space to physical memory by small chunks. Paging will be explained with much fewer details than segmentation. 2.9: Paging, from [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF] Paging is controlled by flags in control registers, mainly by the PG flag (bit 31 of register CR0) that enables or disables paging. Two other flags are involved:

-PEA flag (physical address extension, bit 5 of CR4), allows using 36-bit physical addresses -PSE flag (page size extension, bit 4 of CR4), allows using larger pages. In the standard setting, none of these features are used since they have some downsides that we want to avoid unless these features are really needed.

Without these extensions, a page is 4 Kio (2 12 octets) long. With a 4 Gio (2 32 octets) linear space, we need 2 20 pages to cover the entire linear address space.

The working of paging is mapped in Figure 2.9. Before explaining more precisely, the idea is to cut the linear address in 3 parts: the two firsts are indices in cascading registry of pages while the last part is the offset in the designated page.

Pages are 4 Kio long, and aligned on 4 Kio boundaries (i.e. the first address of each page is a multiple of 4 Kio), so that their base address is always a multiple of 2 12 , i.e. the 12 lower bits are always 0. This 4 Kio alignment constraint is not limited to pages: it is a leitmotiv in everything involved into paging, so that, a lot of lower bits would not carry any information since they are always zero. In order not to waste such precious space, some flags are stored instead, in the same way as the 3 lower bits of segment selectors that store the RPL and the table indicator flag. At the root of paging, there is the CR3 register. It is shown on Figure 2.10. PCD and PWT flags control the caching policy. The page directory is the first step to find a page. As promised, page-directory is 4 Kio-aligned, thus bits 0 to 11 of its base address are 0. This address is physical, because it would be inextricable to resolve a linear address in the structure used for paging.

The page-directory is an array of 32-bit page directory entries. The page-directory can contain up to 1024 (2 10 ) entries and thus, spans over up to 4 Kio. These entries have a structure similar to the CR3 register: 20 upper bits of a base address, followed by some flags (see Figure 2.11). An interesting flag is the P (present) flag: it specifies whether the pointed page-table is currently in physical memory. If cleared, all the other fields are ignored and a system handler is triggered so that the OS can load it into physical memory. This flag is managed only by the system and is very useful to implement transparent swapping. The other flags control page size, access parameters and debugging information. The 10 upper bits of the linear address are used as an index in the page-directory, to get the adequate page-table .  A page-table is very similar to a page directory: it is an array containing up to 1024 32-bit page-table entries. Thus, a page table is also up to 4 Kio long. Page-table entries are very similar to page-directory entries, see Figure 2.12. They also include a P flag, for the same purpose. Here, bits 12 to 31 of the linear address are used to select the right page-table entry. This is the last indirection: the page-table entry contains the physical address of the requested page. Lastly, the 12 lower bits of the linear address are used as an offset in the page to get the physical address.

Memory Isolation

It is important to understand how memory isolation is achieved through paging. We claimed previously that segmentation is often used minimally, and in 64-bit x86, segmentation is almost gone. Segments usually start at address 0 and use all (or almost all) the linear address space, making translation from logical address to linear address trivial. Thus, in this framework, a given logical address, even in distinct processes, is always translated to the same linear address. They are distinguished at paging level: each process gets its own page-directory, and their own page-tables, so that the translation context is different between two tasks, and the same linear address has different images, depending on the task. To modify the paging context, the CR3 register is changed during task-switch. It can be done using hardware task-switch (automatic) or software task-switch (then the system should take care of assigning CR3 correctly). Since paging is different for each task, it is not wise to think of linear addresses as a single space. Each page directory defines a different linear address space, so that each process has its own. These spaces are mostly independent, but they can have some intersection in case of memory sharing.

To avoid catastrophic initiative from a user-level program, assigning CR3 (like any control register) is a privileged operation. Page-directories and page-tables also should not be modified by user programs. Otherwise, they could be changed to point anywhere, allowing arbitrary access to physical memory.

Memory isolation is thus ensured by the system: it should not give the same physical address to multiple pages, with a few exceptions. To share memory, the system needs to assign the same physical address to appropriate linear addresses. Read-only data can also be shared transparently. For read/write data, the same physical address can be shared by multiple linear addresses (even in the same process) when implementing copy-on-write. These few examples illustrate the flexibility of paging and why it became the standard way to manage memory.

Identity Paging

Identity paging is a special kind of paging where a linear address is mapped to the same physical address. It is very useful for the system in several cases:

-In protected mode, when activating paging, the next fetched instruction (pointed by EIP register) is suddenly subject to paging. The physical address of this instruction does not change, but without identity paging, the linear address may map to another physical address. -Creating a task requires to create a new page-directory and some page-tables.

Memory allocation needs to modify these structures. Swapping pages from memory to external storage requires to clear the present flag, and in the other direction we need to set this flag and the new physical memory. Short: paging is a very dynamic mechanism. However, CR3, page-directory entries and page-table entries contain physical addresses. But for the developer, the only address space available is the logic (or linear in the case of trivial segmentation) address space. Without identity mapping, the developer would have to work with physical addresses, even though it cannot access this memory space. In fact, for application tasks, it is reasonably achievable, but it is much more difficult for kernel memory that existed before enabling paging. Even for the first point, identity paging is not a formal requirement. One could make the page containing the current EIP to be located elsewhere in memory but that would require to copy the currently running code to this new page. This is known to be excessively difficult. On the other hand, setting up identity paging on any memory area before enabling paging is really easy: bits 12 to 31 of a page-table entry located in the j th entry of the page table pointed by the i th entry of the page directory should be i × 2 10 + j. To do so, we need to know where is located the current code in memory. Since the code is loaded at the beginning of the boot by some software (see section 2.7 "Boot" (page 37)), we can safely assume that this address is known.

Translation Lookaside Buffers

Paging involves a lot of indirections, and accessing memory is quite slow. To improve execution time, processors have caches to store recently used data and speed up most memory operation by accessing this proxy instead. There is a similar mechanism for paging: pairs of linear and physical address that have been recently computed are stored in translation lookaside buffers (TLBs).

TLBs must be cleared when switching tasks, otherwise, linear addresses would still be translated according to the previous page-directory and tables. In this order, the processor automatically empties TLBs when CR3 is implicitly or explicitly assigned. There is also an instruction to clear specific entries in the TLBs. That can be useful if the system moves a page to another physical address or move it into external storage, making it non-present.

This process seems very transparent and automatic, but it can be controlled in some extent. Especially, the G (global) flag of page-directory entries and page-table entries allows pages not to be automatically invalidated. This can be used to keep pages of kernel memory (that are often useful) quickly accessible. In combination with identity paging, a part of the linear address space is invariably mapped to kernel memory. To prevent user programs to interfere, user code and data segments can be made shorter that 4 Gio, and we can use the available linear space after the end of user segments to store such sensitive data and code. This is an artful combination of segmentation and paging that achieve isolation and fast lookup.

Calls, Interrupts and Exceptions

To build modular software, we need to be able to call a procedure that accomplishes some task and resumes the execution of the previous procedure. This process is a call. This is a very standard notion that exists in almost all programming languages. It is crucial to make it possible and efficient. But calls can be more complex than just calling a procedure belonging to the same program. An important use case of complex calls are system calls: when an application needs to run some privileged code (like allocating memory), the control is given to the OS kernel that does privileged work, and then comes back to the application. Since privilege level is an x86 feature, the processor is involved in this operation. But it is not limited to software, for instance, when a requested page is missing, the processor asks the OS to load the page into memory, and once it is done, the processor gets the control back.

To handle all these kinds of control transfer x86 provides several kinds of calls that address the different scenarios where calls are useful.

Near Calls

The simplest kind of calls are those requested by software and that lead to procedures in the same code segment, they are called near calls (also named intra-segment calls).

Near call is the most minimal kind of call. Apart from some bound checks, it only pushes the current value of EIP on the stack (that is the address of the following instruction) and updates EIP to the target of the call. This instruction has mnemonic CALL.

The converse instruction is RET. It assigns EIP the value of the top of the stack and pops the stack. The execution comes back to the instruction following the CALL.

One could notice that possible parameters of the subroutine are not handled by the CALL and RET instructions. It is up to the software to manage it correctly. There are indeed several ways to pass arguments and to transfer the returned value; such a standard is called a "calling convention". The preferred convention is not a crucial matter, but the caller and the callee have to agree.

The CALL and RET instructions are purely an amenity. There is no guarantee that a CALL will indeed end through a RET. Indeed, there is a programming technique in which all jumps are performed by pushing the desired destination address on the stack and run a RET instruction. The processor is tricked to interpret the top of the stack as a return address and jump to this address. Likewise, after a CALL, one could pop the return address from the stack to replace it with another one, so that RET instruction does not resume the caller execution but end up somewhere else.

Like other horrific tricks, this is not the intended usage of these instructions. But low-level software, such as an OS kernel, makes extended use of this kind of techniques in artful ways.

For instance, Listing 2.3 gets the address of label l, that is the value of EIP at the first line since EIP is the address of the next instruction. Listing 2.4 shows a variant that does not even require a label. The instruction CALL 0 performs a call whose destination is 0 octets away, so it is line 2.

Once again, even if it is not the standard case, one should keep in mind that such tricks exist and will always exist since OS developers have quite good reasons to do so.

Far Calls

Far calls change the code segment. There are two kinds of far calls: to a segment of the same privilege level, or to a segment of a different privilege. The former are intraprivilege calls, and the latter inter-privilege calls. It is discovered dynamically whether a far call is intra-or inter-privilege, thus privilege checks are always performed.

Far calls are, with other similar control flow instructions, the only way to change the CS register.

A Syntactic Digression

Near calls and far calls have different opcodes, thus it must be determined statically (at assembly-time) whether a call is near or far. There are several ways.

Since far calls need to update EIP and CS, it needs 6 bytes of parameters. It can be a literal value (called immediate operand) or a memory operand. The parameter may help to determine the kind of call.

In the case of an immediate value, the 6-byte immediate value allows the assembler to distinguish a near from a far call. For instance CALL label is necessarily a near call while CALL 11:label is a far call using segment selector 11, i.e. index 8 in the GDT (since TI = 0) with RPL = 3 (see subsubsubsection 2.3.2.2.4 "Segment Selectors" (page 23)).

But for a memory operand, it should be specified by other means. And it is especially important because it changes the number of bytes to get from the dereference: as data are not typed, the instruction fixes implicitly the type. Intel's syntax uses the suffix FAR after the mnemonic, so that CALL [EAX] is a near call (and reads only 4 bytes at address EAX), while CALL FAR [EAX] is a far call (and reads 6 bytes at address EAX). AT&T's syntax prefers the l prefix to the mnemonic (e.g. lcall (%eax)).

For consistence, FAR suffix (or l prefix) can be used even with immediate operand. It is even mandatory by not very lenient assembler programs.

But another option is possible (and used): to specify manually the opcode. While very unintelligible, it may be useful in some cases, for instance, when using a very old assembler program that does not have a syntax for far calls, or when the code should be compatible across several assembler programs (or several versions) that use slightly different syntaxes. The trick in this case is to simply dump the bytes of the machine code instruction. The Listing 2.5 gives such an example using GNU assembler syntax. The assembler replaces the label operand by the right address. These pseudo-instructions write 7 bytes corresponding to a far call with immediate operand (opcode = 9AH) to the target label with segment selector 11.

Intra-Privilege Calls

A far call to a segment of the same privilege is still quite simple. After mandatory privilege checks, the processor pushes caller's CS and EIP on the stack, then load the given segment selector in CS and the target instruction in EIP.

The converse operation, far return, reads the caller's EIP and CS on the stack and thanks to privilege checks, it establishes that it is an intra-privilege return and just has to load EIP and CS.

Like for near calls, one should not lack prudence for the exact same reasons: far calls and far returns are not necessarily well-parenthesized, calling convention is up to the software, and far calls can be used for other purposes than the intuitive one. Inter-privilege calls are far calls whose target segment has a different privilege than the caller task. However, this is not permitted in all cases. In particular, direct call to a non-conforming code segment is allowed only within the same privilege-level and to conforming code segment from numerically higher privilege (less privileged). For conforming code segment, the CPL does not change, so both these mechanisms do not allow privilege level to be changed. Indeed, it would be risky to allow tasks to climb privilege levels without control.

Inter-Privilege Calls

CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING

Fortunately, x86 provides a protected way to execute a procedure with higher privilege: call gates. Call gates are system descriptors (and thus, lie in the GDT) that points to another procedure. Their structure is shown on Figure 2.13 which is very close to the structure of a segment descriptor (see Figure 2.7 "Layout of a segment descriptor" (page 20)). Call gates contain a segment selector for the segment containing the target procedure. This segment is usually more privileged than the CPL. The call gate also contains the target (offset in the segment) of the procedure, that is, the address of the first instruction. DPL field specifies the minimum (numerically larger) privilege to access this call gate. There is an extra field that has no equivalent in any other kind of descriptor: parameter count. It is the number of parameters to copy from the caller's stack to the callee's stack if a task switch occurs.

Since call gates carry the offset of the target procedure, one cannot call arbitrary code with high privilege. So, as long as the target procedure is reliable, this kind of privilege transfer is safe. We can also notice that the offset part (non-segment selector) of the operand of the instruction is ignored, although it is still required to have a wellformed instruction. So, one can choose any arbitrary value, and typically 0. Since 0 is rarely a desired offset, this helps to point out far calls to call gates.

An important limitation, often untold, of call gates is that they only allow calls that transfer privilege to a higher level. The rationale is that less privileged code is untrustworthy and thus, should not be called from higher privilege. This is generally true, but sometimes it is useful to decrease privilege level. This process will be explained in section 2.7 "Boot" (page 37).

An inter-privilege call, in addition to change CS and EIP requires to change the current stack for a higher privilege stack. SS and ESP are thus saved as well, to be restored later. Some other steps and checks are required but are out of our scope.

Interrupts and Exceptions

Interrupts and exceptions are forced control transfers between the currently running task to a specific function (the handler). Interrupts and exceptions are really close but differ in their origin. Usually, interrupts are external and asynchronous, while exceptions are triggered when an abnormal condition is detected in an instruction execution.

There are 3 kinds of exceptions: faults, traps and aborts.

-A fault allows the processor to resume the execution of the faulting instruction. This is useful in situation where the fault can be corrected. For instance, if a page is missing in physical memory, it can be loaded, and the faulting instruction can be correctly executed. Thus, a fault must be handled before the faulting instruction. Since x86 does not implement time travel, the processor simply restores the machine state to the precondition of the faulting instruction before handling the fault and retrying to run the instruction. -On the other hand, traps are handled after the trapping instruction. Traps are used when the instruction is completed correctly but something else must be done in addition. Debugging instructions are traps whose purpose is to transfer control to debugging functions.

-Aborts, which form the last category, do not always provide their origin and thus do not allow resuming execution. Aborts report catastrophic situations, such as corrupted system structures, when there is no hope of recovery. Interrupts, like traps, are handled after the current instruction.

Each interrupt and exception have an identifier between 0 and 255. Identifiers between 0 and 31 are dedicated to exception (and a special interrupt). Some identifiers of this range are not used, but they are reserved. Identifier from 32 to 255 are available for user-defined interrupts.

Most of the technical details are not relevant in our context. The interesting part is control transfer. Interrupts and exceptions are so similar (especially in our scope) that in the following, statements about one of them will apply to both, except if specified otherwise. To register a handler for an interrupt, there is a special structure very similar to the GDT called IDT. Like GDT, it contains descriptors, but only among 3 kinds of system descriptors: task gates (see Figure 2.14), interrupt gates (see Figure 2.15) and trap gates (see Figure 2.16). The two latter are very close to a call gate. D flags specify whether CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING the destination code segment is 16-bit (if cleared) or 32-bit (if set). The difference with a call gate is that there is no parameter count field, since there is no parameter. Trap and interrupt gates differ in the way they manage interrupts that are triggered during the handling of another interrupt. Task gates are pointers to a TSS descriptor in the GDT which are pointers to a complete procedure with context. Tasks will be explained in section 2.5 "Tasks" (page 34).

To get the right gate for the i th exception, the processor reads the i th descriptor of the IDT, i.e. the entry with offset of 8i bytes. The address of the IDT is stored by the processor in the system register IDTR, which has the same structure as GDTR (see subsubsubsection 2.3.2.2.3 "Global Descriptor Table" (page 23)), that is a 32-bit base address and a 16-bit limit. IDTR is assigned using LIDT instruction, just as GDT is assigned by LGDT. IDT, as well as GDT is designed to be a long-lived structure.

When an exception is handled, the stack of the handler contains CS and EIP of the interrupted task, the value of EFLAGS register at the point of exception, and, if a privilege change happens, SS and ESP or the interrupted task. Moreover, depending on the exception and how it was generated, the stack may contain an error code. The content of the stack allows the handler to retrieve the context and act properly. To return from interrupt handler to the interrupted task, x86 provides the IRET instruction.

Tasks

Tasks are the elementary structures that x86 processors use to handle multitasking. A task can achieve a high-level work (like a process) as well as a low-level one (like an interrupt handler). A task is described by a task-state segment (TSS) that saves the current state of a task when a task switch occurs, to be able to restore it later.

A task can be executed either explicitly (by using a jump or a call) or implicitly (as an exception-handler).

Task-State Segment

The TSS contains two kinds of fields: dynamic and static. Dynamic fields are updated when a task is suspended, so that they can be used to restore the current state. Dynamic fields include the value of registers that may be changed by the user (general-purpose registers, segment registers, EFLAGS and EIP) and a link to the previous task (to allow task nesting). Static fields are set when the task is created and the user does not have the privilege to change them. Among others, they store the value of CR3 register (address of page-directory see subsubsection 2.3.3.1 "Translation Mechanism" (page 25)) and higher-privilege stacks (segment selectors and stack pointers) to use when a privilege change occurs (e.g. because of an interruption handling or a call to a call-gate). It is worth noticing that the memory space of a task is not contained in its TSS: it stays in the memory. Indeed, we need to save registers because other tasks will overwrite them, but if memory management is correctly implemented, the memory spaces of tasks are disjoint. During the execution of a task, dynamic fields are not updated, and they

TASKS

35

probably do not match their respective registers; they are written only on task switch. The TSS is technically a segment, even it has a very precise structure and size. This means that it has a descriptor in the GDT, called a TSS descriptor. Tasks may also be accessed through a task gate. Task gates are safe pointers to a TSS descriptor (see Figure 2.14 "Structure of a task gate" (page 33)). They might be in the IDT so that a task can be used as an interrupt (or exception) handler.

A TSS descriptor is shown on Figure 2.17. Flags have the same meaning as for data segment descriptor (see subsubsubsection 2.3.2.2.1 "Segment Descriptor" (page 19)).

Task Register

When a task switch occurs, the processor must be able to locate the TSS of the currently active task, to know where to save the current context. During a privilege change (callgate or interrupt handler), the processor must get the privileged stack to use. This is also stored in the TSS of the currently executing task.

To be able to get the current TSS, there is a special register that keeps the TSS selector, in the same way as segment registers keep their respective segment selectors. This segment is called task register (TR). Like segment registers, TR is made of a visible part that holds the selector and which is the only part accessible to software, and a hidden part which is managed by the processor and stores the content of the TSS descriptor.

The task register must be loaded explicitly at least once (to set up the first task). This is done using the LTR privileged instruction. After that, the task register may still be managed manually, but it will be automatically updated during task switch.

Using Tasks

Tasks are not limited to be independent processes: they also can be linked. When a task calls another, the old task is saved, the new task is promoted to be the active task and a link to the old task is stored in new task's TSS. The link allows returning from the new task to the old one, like a return from a call would. Iterating this process allows nesting arbitrarily many tasks, though each task can only appear once in the list. Indeed, a task that is pointed by another task is marked as busy and, to avoid nasty recursion, calls to busy tasks are forbidden.

As powerful tasks can be, tasks are very rarely used beyond minimal requirement. Though x86 provides task switching facility using x86 tasks, it can also be done at the software level, by the system, by modifying the current task so that is contains the state of the new task. Indeed, this is the preferred way, since hardware context-switches have two main flaws.

The first drawback of hardware task switching is the lack of portability. This variant of context switching is very specific to x86 and it would imply an additional cost to adapt a kernel to an architecture without this facility.

The second problem of hardware context switching is more linked to the modern usage of protection features. We have already seen that segmentation is minimally used. In particular, since paging does most of the work, all unprivileged tasks use the same code and data segments. Moreover, we do not use LDT, privileged stacks are the same for all tasks, and so on. But a hardware context switch saves, uselessly, all these parameters, performs checks on the new values and assigns them. With modern usage of protection features, this is a waste a time. Software context switching saves no more that required and performs fewer checks since there are fewer risky assignments. As surprising as it might be, software context switching is usually faster since it performs much fewer instructions.

System Calls

A system call (abbreviated as syscall) is a request made by an unprivileged program to ask the system a job that requires privileges, and possibly knowledge about the system state. Syscalls are used for a large variety of tasks, for instance allocating or sharing memory, creating process, device access (especially to access files), sending signals, or setting up timers.

Triggering a System Call

There are many ways to transit to high privilege. They all have advantages and drawbacks. The very common trade-off is portability vs. performance and security.

A common way to implement system calls is to use interruption. One chooses an arbitrary user-defined interrupt vector (in Linux, for instance, it is 80H) and set up the correct handler for this vector in the IDT. Parameters are passed through registers, in particular, one of them (usually (E)AX) stores the code for the desired syscall. The interrupt handler is a mere dispatcher that calls the appropriate function, according to the code. Return to the user application is done thanks to an interrupt return (IRET). If applicable, general-purpose registers holds result values. This method is relatively portable and the standard way with 32-bit x86.

Rather that going through an interrupt, one can directly use a call gate. This is more architecture-specific: while some kind of interrupts exists in a lot of architectures (requiring only to adapt the handling), call gates are more particular to x86. Moreover, to do the actual syscall, we need to know where the call gate is placed in the GDT.

Some architectures, such as 64-bit x86, include a specific operation designed to travel between user and kernel land. These instructions are usually preferred when available.

Not only this kind of instructions becomes more and more widespread, but since they are specifically designed for implementing syscall, they are more efficient because they take the liberty of not providing useless flexibility.

There are some other possibilities, but largely minor and not detailed here

Passing Arguments

To pass parameters to a syscall, there are mainly 3 solutions: using registers, stack or memory. Registers are really fast, but provide very limited space and force the caller to save them if it needs their previous contents. Stack is not limited, compatible with most C calling conventions and allows nested syscalls, but it is not as fast as registers.

Memory is even more unlimited but requires one of the previous methods to pass the pointer to the parameters. With none of these methods, the syscall handler has guarantees that the caller passes the appropriate number of parameters. Thus, the system must be really careful: checking that integers are in allowed ranges, pointers are valid, sizes are non-negative, and so on. It is important that even an illegal call does not disturb other processes and does not compromise system integrity.

Boot

When powering up the machine, firmware finds the starting point for the OS. We are not interested in the specification of these very first steps, but rather in what happens once this entry point is loaded into memory and it starts running.

At this point, the processor is usually in real-address mode. On some machines, the firmware takes the initiative to enable protected mode. This may look appealing, but it might make some unfortunate choices. If this is not done already, the first step is to set up segments (at least kernel segments) and enable protected mode by setting the right flag (PE (Protection Enable), bit 0 in CR0). To complete protection activation, one should use a far jump to set the current code segment. Since real-address mode is only able to address 1 Mio of memory, it might be difficult to set up all the kernel structures this early. It is wiser to transit to protected mode before, allowing addressing 4 Gio.

Once protected mode is enabled, the addressing space is drastically extended. We can serenely setup big structures directly at the right address. We need especially to set up the IDT and paging. The real challenge it to get into ring 3. If they were not defined before, we need data and code segments for user-level tasks and at least one TSS (as explained in section 2.5 "Tasks" (page 34)). We should also have taken care of providing a way to make syscalls. This last point is not a formal requirement, but a system without syscall would be very limited. Changing to ring 3 is a difficulty per se. In x86, there are ways to call higher privilege code with current privilege, or to call a specific procedure with high privilege, but, since it is not a good idea to run unprivileged (so untrusted code) from high privilege, x86 does not provide any facility to lower the privilege level. The solution is to trick the processor to make it believe we used to be CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING in ring 3. One artificially builds a stack as it would be after a privilege change from ring 3 to ring 0 and instructs the processor to perform a return from inter-privilege call, making it arrive into ring 3.

The first process to be called is the initialization process (like init in Unix-like OSes) whose task is to launch other processes. Once the initialization program is started, the system has a much more discreet role: system functions are called mainly by syscalls or through interrupt handlers, but they are not constantly running. An interesting example is the scheduler.

The scheduler is the part of the kernel that chooses the task to execute next, and performs task switch. It might be called from anywhere in the kernel, but we need to be able to trigger the scheduler directly, and not wait to get into kernel land, otherwise it might not be called for an arbitrary long time. This is achieved by configuring a timer, an external piece of hardware that triggers an interrupt at given frequency. When this interrupt is detected, the processor will call the given handler installed by the system. This handler is an entry point that triggers rescheduling.

Overall, the kernel is just waiting in the limbo of IDT, GDT, and similar structures, waiting to be called upon for a syscall, or an interrupt or exception handling.

Short, we have presented the x86 architecture and, in particular, memory protection features. We have also explained how to set them up and use it in a modern operating system.

Chapter 3

Introduction

W hen writing low-level source code, like an OS, we need to use assembly code to control things that are beyond the model of C. Since C aims to be platformindependent, everything about processors features must be done in assembly. For instance, in the framework of system development on x86, this includes the setup of segmentation, paging and IDT.

However, assembly is not really friendly for the developer: there is no high-level control flow (like while loops or "else if" branching), no complex expressions, no types and no scopes. Thus, for all the parts that do not really need to use assembly code, we prefer using a high-level language. Among other constraints, the chosen high-level language must have an extended control on memory allocation, be able to manage data structures with specific bits arrangements and interface nicely with assembly code. Because of these conditions, and the weight of history, a common choice is C.

Linking objects files coming from C and assembly is already nice, but most compilers go even further and allow writing small piece of assembly in the body of C functions. Otherwise, we would need to keep the assembly in separate files and perform calls to jump between C and assembly parts. The first issue of keeping languages separated is a readability problem (having to look in another file for a small snippet of assembly that makes no sense alone), and the second is that it might pull down performance by performing a call/return rather than just executing a linear sequence of instructions.

Such inclusions of piece of x86 assembly in C are quite harmless if they just perform simple actions like loading a segment register, but in practice, such assembly snippets are used to do much more complex operations, including complex control flow instructions. This is a major problem since C and assembly have very different control flow structures. Inline assembly blocks may also modify variables that are visible from C (and not only registers), which may be difficult to understand since assembly and C have very different memory models.

In this part, we will study the semantics of mixed C and x86 assembly source code with very few unnecessary hypotheses. Indeed, using abstract interpretation on such code requires a notion of soundness that is defined with respect to the semantics.

Due to the complexity of C, we will not directly work with it, but rather with a subset that contains interesting features. The first step will be to define this language and its semantics. Similarly, we will define the interesting part of the x86 assembly language and its semantics. Given these languages, we will see how they are mixed from the developer point of view. Then we will study complex but realistic examples that justify some choices for the following and show that we cannot make a lot of hypotheses, even among reasonable ones. Once we have defined all the background and have all the constraints, we will propose a semantics for mixed C and assembly. In a final chapter, we will address an ambiguity problem caused by the Intel's syntax of assembly.

Chapter 4

A Minimal C-like Language L et us define the C-like language used in the following. It looks like a small fragment of C, but it already includes some difficulties like pointers and functions. We name it Asclepius for Architecture-Specialized C-like Language Expressly Planned to Interact with Underlying System, and to evoke the hero and god of medicine (Ἀσκληπιός) in ancient Greek mythology: this language heals some aspects of C that are problematic for our usage * . Discussion about the main differences with C can be found in section 4.4 "Comparison with Real C and Thoughts" (page 58).

Syntax

We give the syntax from the top, using the extended Backus-Naur syntax (ISO/IEC 14977:1996). In this syntax, {group} stands for repeated groups and [group] for optional groups. We defined top-level declarations; they include functions and global variables. The only types we consider are integer types or any depth of pointer to an integer type. Variable names, called name in the syntax, belong to an arbitrary set, assumed to be big enough, denoted by V, and functions names ( f un_name ) are from a disjoint set denoted F. The last non-terminal symbol to define is c_stat. It corresponds to statements, which are the operational part of the language: they make effective computations. The i subscript on the * of lvalues stands for the number of bytes dereferenced. It is useful to be able to size any lvalue. Indeed, name has a size determined by the declaration of the variable. It still allows overriding the size through a simple trick: for instance, if x has size 4, but we want to write only 2 bytes at its address we can do * 2 &x.

Notation 4.1 -Size of lvalues

For a lvalue l, we denote |l| the size of the lvalue.

LABELS

45

Notation 4.2 -Set of statements

We denote Stat C the set of statements.

We will not define the non-terminal op , because it would be much ado about nothing. But we make some assumptions. They are pure operators: unlike some C operators (like = or +=), they have no side effect. To stay on the safe side, function calls are excluded, even calls of pure functions. We allow any arithmetic or logic operator we want, whatever the arity. To interpret such an expression as a Boolean value, we do the same as in C: 0 is false, everything else is true. For instance, legal operators include unary minus (unary operator), addition (binary operator) or ternary expressions (like C's cond ? t : f).

Labels

To be able to mix finely C and assembly, we want to have a precise description of the evaluation of C code, that is an operational semantics. The selected formal framework is a transition system-based semantics. A transition system inventories all possible configurations and connects them with valid transitions. Any path in the transition system is a correct execution of the program. Configurations must contain two pieces of information: the state of the memory and the current point of the program. We thus need a way to specify a position in a program.

To indicate program points, we use labels. A frequent solution is to insert labels into the syntax of the language, but this solution is in fact quite hard to deal with. It is difficult to adapt these rules such that each statement is surrounded by labels, while avoiding duplicates (multiple labels at the same program point). Another approach to annotate each component of the program is to give axioms about labels for each component of the program. This part is largely inspired by [START_REF] Cousot | The Calculational Design of a Generic Abstract Interpreter[END_REF] (section 12 "Semantics of Imperative Programs" (page 40)).

In our context, statements belong to functions. They are not organized as a tree but as a forest where each function is the root of a tree.

Definition 4.4 -Paths

A path π = (f, p) is an ordered pair where f ∈ F is a function name and p ∈ i∈N N i is a finite sequence of integers. The set of paths is denoted Π. We define the concatenation function . : Π × N → Π that extends the sequence by (f, (p 0 , . . . , p n )).i := (f, (p 0 , . . . , p n , i))

where ε is the empty sequence. Lastly, we define

Cmp : P C → P (Stat C × Π) P → f ∈F[P ] Cmp P (f )
Components are not easily usable: we are rather interested in adjacent statements rather by their path from the root of the function. But we can use these paths to have a context-sensitive definition of labels, thus not giving the same label to identical statements with distinct positions in the program.

We introduce L C , the set of all labels. It is disjoint from any previously introduced sets. Given a program P ∈ P C , we are interested in two functions at P : Cmp (P ) → L C after P : Cmp (P ) → L C but for the sake of formalization, we also need a third function:

in P : Cmp (P ) → P (L C )
We will not define these functions, but give axioms on them and assume such functions exist. However, they can be built using a syntax-directed positioning of labels and smashing rules to identify labels that are at the same place.

Axiom 4.1 -Labeling of programs

We are given a program P ∈ P C . We globally have ∀C ∈ Cmp (P ) , at P (C) = after P (C) Moreover, for C ∈ Cmp (P ) depending on the case:

-

If C = (if (c) S, π) in P (C) = {at P (C) , after P (C)} ∪ in P ((S, π.1)) ∧{at P (C) , after P (C)} ∩ in P ((S, π.1)) = ∅ -If C = (while (c) S, π) in P (C) = {at P (C) , after P (C)} ∪ in P ((S, π.1)) ∧{at P (C) , after P (C)} ∩ in P ((S, π.1)) = ∅ -If C = ({S 1 . . . S n }, π) at P (C) ∈ n i=1 in P ((S i , π.i)) ∧ after P (C) ∈ n i=1 in P ((S i , π.i)) ∧ in P (C) = {at P (C) , after P (C)} ∪ n i=1 in P ((S i , π.i))
∧∀i ∈ 1, n -1 , after P ((S i , π.i)) = at P ((S i+1 , π.i + 1))

∧∀i ∈ 1, n -1 , in P ((S i , π.i)) ∩ in P ((S i+1 , π.i + 1)) = {after P ((S i , π.i))} ∧∀i ∈ 1, n , ∀j ∈ i + 2,

Semantics

The next step toward a definition of a semantics is to choose the memory model.

Memory Model

To simplify the semantics, we will make a few assumptions that do not change the expressive power of the language. First, though we have some scopes, we forbid shadowing.

Even with this restriction, scopes are not useless: they still allow variables to have a limited lifetime. We also consider that pointers are 4-byte integers.

Notation 4.4 -Size of types

For a type t, we denote |t| the size of a variable of type t.

This mimics the notation for sizes of lvalues (see notation 4.1 "Size of lvalues" (page 44)). It is good to note that types are not very useful in our language: they just give hints to the developer and specify sizes of variables.

Since we use pointers and variable names, a good guess is to use a memory model made of a pair of a stack of environments and a heap: the environments map names to addresses and the heap maps addresses to values. We use a stack of environments to handle correctly function calls, during which some variable names became momentarily unknown. We also need an additional environment for global variables. Formally, we let A := 0, 2 32 -1 the set of addresses, I := 0, 2 8 -1 the set of potential values of a byte, and

E C := V A H C := A I M C := E C + × L C × E C × H C
with some restrictions: given

(v i ) i∈ 1,n , (p i ) i∈ 1,m , g, h ∈ M C then supp (h) = Im (g) ∪ n i=1 Im (v i ) n = m + 1
where supp (h) is the support of h and Im (g) is the image of g (see section A.5 "Function Theory" (page 374)), and

∀(a, b) ∈ ({v i | i ∈ 1, n } ∪ {g}) 2 , ∀(x, y) ∈ V 2 , a(x), a(x) + |x| -1 ∩ b(y), b(y) + |y| -1 = ∅ ⇒ a = b ∧ x = y
(variables have distinct memory location). Moreover, there are some restrictions that are more contextual. Global variables can only appear in the third component of a memory state. Each map of the first component is only about local variables of a single function.

Let us explain how the parts of a memory state will be used. The first part is a stack of environments: they store the addresses of the local variables of functions in the call stack. The second component is the stack of return addresses. They allow jumping back when a return is encountered to the function call. The third component is the environment of global variables. This one is constant along all the execution. The last part is the heap: it represents the mapping from addresses to values. This is where values are actually stored.

The first and second components are modified by calls and returns: calls append a local environment and a return address, and returns pop them. Values on these stacks cannot be modified in-place. The third component is immutable: it is nondeterministically chosen in the initial state and is not modified in the execution. The fourth component is very mutable: it can be modified by any assignment, for instance.

Transition Systems

We use [START_REF] Cousot | Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints[END_REF]-style semantics: based on a transition system. It could be written a → b if τ is clear in the context.

Definition 4.6 -Transition system

A transition system is a 3-tuple (Q, I, τ ) where -Q is a set of state (non-necessarily finite), -I ⊆ Q is the set of initial states, -τ ⊆ Q × Q is the transition relation.
The transition system of a program has Q = (L C ×M C ) Ω and its transition relation is defined by induction over the syntax. Here Ω is a set of possible errors that can occur during the execution.

Semantics of Lvalues and Expressions

We need the semantics of lvalues and expressions. Since lvalues can be the left-hand side of an assignment, we are interested in the address they point to and not the pointed value: we need to know where to write, not what will be overwritten.

CHAPTER 4. A MINIMAL C-LIKE LANGUAGE Definition 4.7 -Address of a variable Let n ∈ N, (v i ) i∈ 1,n ∈ E C n and h ∈ E C . We define get_address (v i ) i∈ 1,n ,g : V A l → v n (l) if l ∈ supp (v n ) (local variable) g(l)
otherwise (global variable)

the function that returns the address of a variable.

The map get_address (v i ) i∈ 1,n ,g is a partial function whose support is the union of support of v n and g. This is what we expect since a well-formed program only reference visible variables, i.e. variables who belong to the support of get_address (v i ) i∈ 1,n ,g . This is a syntactical property, so it is a reasonable assumption. We can also remark that variables of deeper functions are not visible.

Definition 4.8 -

Dereferencing Let h ∈ H C . We define deref h : Z × N * → P (Z) × P (Ω) (a, size) →          (∅, {ω a }) if a, a + size -1 ⊆ A (∅, {ω a }) if a, a + size -1 ⊆ supp (h) size-1 i=0 h(a + i) • 2 8i , ∅ otherwise
where ω a is the error raised by an illegal memory access. We extend it to the power set:

deref h : P (Z) × N * → P (Z) × P (Ω) (A, size) → a∈A π 1 (d) , a∈A π 2 (d)
where d = deref h (a, size)

The first case of the definition of deref h is in fact redundant since supp (h) ⊆ A, but they are intuitively two different kinds of errors: in the first case, the address is outside bounds, while in the second, the address might be valid, but is not in this particular heap.

We can note that we only get non-negative integers by dereference. If we need a signed integer (for instance, represented by two's complement), we just have to wrap the dereference by a unary operation that performs the conversion.

Notation 4.6 -Semantics of operations

Given an n-ary operation op, we denote op : (P (Z) × P (Ω)) n → P (Z) × P (Ω) is the semantics of op.

It is interesting to remark that op is able to remove some errors. This is useful to mimic the behavior of C's ternary operator, or short circuit semantics of logical operators. For instance x = 0 ? 1 : 1/x does not cause a division by 0. But in our semantics, 1/x is computed even if not necessary, so we need to make the error disappear if needed.

Definition 4.9 -Semantics of a lvalue and expression

We define the semantics of lvalues and expressions by mutual recursion. The semantics of an lvalue l is

⦉l⦊ : M C → P (A) × P (Ω) m = (v, p, g, h) → (e) (m) if l = * i e get_address v,g (l) , ∅ if l ∈ V
and the semantics of an expression e is

(e) : M C → P (Z) × P (Ω) m = (v, p, g, h) →                  ( a, b , ∅) if e = [a,b] ⦉l⦊ (m) if e = &l    (V, O ∪ O ) where (V, O ) = deref h (A, |e|) where (A, O) = ⦉e⦊ (m) if e = lvalue op ((e 1 ) (m) , . . . , (e n ) (m)) if e = op(e 1 , . . . , e n )
Since these functions are defined inductively on the structure of expressions, they are well-founded.

Semantics of Statements

Let us start by a few notations that will come in handy very soon.

Notation 4.7 -Variables at program point

Given a label l, we denote V [l] ⊆ V the set of variables that can be alive at this program point, that is the set of variables declared in current and outer scopes, including global definitions.

CHAPTER 4. A MINIMAL C-LIKE LANGUAGE

It is good to remark that variables declared in the body of a while-loop can be alive at the beginning of the block, since they are not killed between the first and the second iteration.

Notation 4.8

Given a memory state ρ = ((v i ) i∈ 1,n , p, g, h) ∈ M C and a variable set V , we denote ρ |V the restriction of visible variables in ρ to V , that is

v 1 , . . . , v n-1 , v , p, g , h |Im(v )∪Im(g ) where v = v n|V g = g |V
and f |A is the restriction of the function f to the support A (see section A.5 "Function Theory" (page 374)).

Notation 4.9 -Memory state on given support

Given a variable set V ⊆ V, we denote M [V ] the set of memory states whose set of living variables is V , that is

((v i ) i∈ 1,n , p, g, h) ∈ M C supp (v n ) ∪ supp (g) = V

Transition Relation

Now we have the semantics of expressions, we can define the semantics of programs. We will define a function that returns the transition graph generated by a statement in a program. We recall that the set of states of the transition system is

(L C × M C ) ∪ Ω.
Ω is a set of errors. Once the system falls into such a state, we consider that the program stops. Consequently, there is no transition whose left-hand side is in Ω. We denote τ

P C a S b ∈ P ((L C × M C ) × ((L C × M C ) ∪ Ω))
the set of transitions generated by the statement a S b in program P . We have already explained that this definition is inductive, now we detail the rules for each kind of statement.

Blocks

For a block, there are 2 special transitions: one that enters the block, without changing the memory state, and one that exits the block. This exiting transition is a bit more complicated since it kills variables that fall out of scope. Moreover, we add all the transitions generated by the statements in the block.

τ P C l { a 0 S 1 a 1 . . . a n-1 S n an } m = {(l, ρ) → (a 0 , ρ) | ρ ∈ M [V [l]]} ∪ (a n , ρ) → m, ρ |V[m] ρ ∈ M [V [a n ]] ∪ n i=1 τ P C [ a i-1 S i a i ]
If the block is empty, there is no a i label, thus we simply short circuit the block.

τ P C l { } m = {(l, ρ) → (m, ρ) | ρ ∈ M [V [l]]}

Conditional Branching

The semantics of branching statement is not surprising. We have transitions to go from the beginning of the statement to the body for memory states for which the condition may be true (non-zero). Contrariwise, there are transitions that shortcut the branching when the condition can be false (zero value). In addition, if the condition returns errors, appropriate transitions are added. There are also transitions to exit the body of the statement and the transitions corresponding to the statement.

τ P C l if(e) a S b m = {(l, ρ) → (a, ρ) | ρ ∈ M [V [l]] , Z * ∩ π 1 ((e) (ρ)) = ∅} ∪ {(l, ρ) → (m, ρ) | ρ ∈ M [V [l]] , 0 ∈ π 1 ((e) (ρ))} ∪ {(l, ρ) → ω | ρ ∈ M [V [l]] , ω ∈ π 2 ((e) (ρ))} ∪ τ P C a S b ∪ (b, ρ) → m, ρ |V[m] ρ ∈ M [V [b]]

Loop Statements

This is also very straightforward. The main difference with the case of the conditional branching statement is that exiting transitions lead to the head of the loop to test the condition again. It is interesting here to remark that local variables declared in the body of the loop are not killed at the end of the block since they can be alive everywhere in the loop, typically after the first iteration. However, they are killed when quitting the loop since these local variables fall out of scope at this point.

CHAPTER 4. A MINIMAL C-LIKE LANGUAGE

τ P C l while(e) a S b m = (l, ρ) → (a, ρ) ρ ∈ M [V [l]] ∪ M [V [b]] , Z * ∩ π 1 ((e) (ρ)) = ∅ ∪ (l, ρ) → m, ρ |V[m] ρ ∈ M [V [l]] ∪ M [V [b]] 0 ∈ π 1 ((e) (ρ)) ∪ {(l, ρ) → ω | ρ ∈ M [V [l]] ∪ M [V [b]] , ω ∈ π 2 ((e) (ρ))} ∪ τ P C a S b ∪ {(b, ρ) → (l, ρ) | ρ ∈ M [V [b]]}

Assignments

This statement modifies the appropriate bytes in the heap. The first byte to be modified is given by the lvalue and the number of modified bytes is the size of the lvalue. There are also all the required error transition to handle cases where the evaluation of the left-hand side or right-hand side of the expression may fail.

τ P C l x = e; m =              (l, ρ) → m, (v, p, g, h ) ρ = (v i ) i∈ 1,n , p, g, h ∈ M [V [l]] , h ∈ H C : ∃y ∈ π 1 ((e) (ρ)) : ∃a ∈ π 1 ⦉x⦊ (ρ) : h = h [a + i → y [8i : 8i + 7]] |x|-1 i=0              ∪ {(l, ρ) → ω | ρ ∈ M [V [l]] , ω ∈ π 2 ((e) (ρ))} ∪ (l, ρ) → ω ρ ∈ M [V [l]] , ω ∈ π 2 ⦉l⦊ (ρ)

Declarations

This statement is much more tricky. The simplest case (last in the formula) is the situation where the variable already exists (typically, in a loop). In this case, the memory state is not modified. If the variable does not exist, it is more complicated. We need to find an address a such that the memory block of the appropriate size is free. The heap is updated with arbitrary bytes and the current local environment is updated to make the name point to the address a.

If there is no memory block large enough to store the variable, we transit to a special error state ω o (out of memory).

τ P C l type name; m =                                              (l, ρ) → m, ρ ρ = (v i ) i∈ 1,n , p, g, h ∈ M [V [l]] , ρ ∈ M [V [m]] : ∃a ∈ A : ∃(b i ) i∈ 0,|type|-1 ∈ I |type| : ∃v ∈ E C : ∃h ∈ H C : a, a + |type| -1 ⊆ A, a, a + |type| -1 ∩ supp (h) = ∅, h = h [a + i → b i ] |type|-1 i=0 , name ∈ supp (v n ) , v = v n [name → a] , ρ = v 1 , . . . , v n-1 , v , p, g, h                                              ∪                  (l, ρ) → ω o ρ = (v i ) i∈ 1,n , p, g, h ∈ M [V [l]] : name ∈ supp (v n ) , ∀a ∈ A, a, a + |type| -1 ⊆ A ⇒ a, a + |type| -1 ∩ supp (h) = ∅                  ∪    (l, ρ) → (m, ρ) ρ = (v i ) i∈ 1,n , p, g, h ∈ M [V [l]] : name ∈ supp (v n )   

Return Statements

To come back from a function, we read the return address (as a label) in the memory state (the top of the second component). Thanks to this label, we can know which statement has made the call to the current function. From the statement, we get the variable in which the returned value must be written. We get the address of this variable in the local environment of the calling function (so the second topmost in the stack), and we update the heap accordingly. Finally, the transition pops the stacks of local environments and return addresses and jump to the return address.

If the return expression evaluates with an error, appropriate error transitions are added.

If the stack of local environments has size 1, this means there is no calling function: this is the end of main function, and thus the end of the program. In this case, no transitions are generated and the execution end in a regular (non-error) state.

CHAPTER 4. A MINIMAL C-LIKE LANGUAGE

τ P C l return e; m =                                                (l, ρ) → ret, ρ ρ ∈ M [V [l]] , ret ∈ L C , ρ ∈ M [ret] : ∃x ∈ V : ∃π ∈ Π : ∃y ∈ Z : ∃r ∈ A : ∃(h, h ) ∈ H C 2 : ∃n ∈ N * : ∃g ∈ E C : ∃(v i ) i∈ 1,n ∈ E C n : ∃(p i ) i∈ 1,n-1 ∈ L C n-1 : ρ = (v i ) i∈ 1,n , (p i ) i∈ 1,n-1 , g, h n 2 ∧ ret = p n-1 ∧ y ∈ π 1 ((e) (ρ)) , after P ((x = f ( . . . ), π)) = ret, r = get_address (v i ) i∈ 1,n-1 ,g (x), h = h [r + i → y [8i : 8i + 7]] size-1 i=0 , ρ = (v i ) i∈ 1,n-1 , (p i ) i∈ 1,n-2 , g, h |V[ret]                                                ∪ {(l, ρ) → ω | ρ ∈ M [V [l]] , ω ∈ π 2 ((e) (ρ))}
where size is the size of the return type of the current function.

Function Calls

This statement is the more complex but not complicated. We push a new local environment on the stack. This new environment is initialized with the values of parameters, and the heap is updated accordingly.

If there is no more room in the memory, the transition leads to error state ω o .

If any of the parameters of the function call may evaluate into an error, transitions to these errors are added.

τ P C l x = f (e 1 , . . . ,e n ); m =                                                                  (l, ρ) → c, ρ ρ = (v i ) i∈ 1,n , (p i ) i∈ 1,n-1 , g, h ∈ M [V [l]] , c ∈ L C , ρ ∈ M [c] : ∃(a i ) i∈ 1,n ∈ A n : ∃(range i ) i∈ 1,n ∈ P (A) n : ∃v n+1 ∈ E C : ∃h ∈ H C : ∃v ∈ E C n+1 : ∃p ∈ L C n : ∀i ∈ 1, n , range i = a i , a i + |x i | -1 , ∀i ∈ 1, n , range i ⊆ A ∧ range i ∩ supp (h) = ∅, ∀i ∈ 1, n , range i ∩ i =j range j = ∅, ∀i ∈ 1, n , v n+1 (x i ) = a i , ∀i ∈ 1, n , b i ∈ π 1 ((e i ) (ρ)) , h = h [a i + j → b i [8j : 8j + 7]] n i=1 |x i |-1 j=0 , v = (v 1 , . . . , v n , v n+1 ) , p = (p 1 , . . . , p n-1 , m) , ρ = v , p , g, h , c = at P (Body P (f ))                                                                  ∪                        (l, ρ) → ω o ρ = (v i ) i∈ 1,n , (p i ) i∈ 1,n-1 , g, h ∈ M [V [l]] : ∀(a i ) i∈ 1,n ∈ A n , ∀(range i ) i∈ 1,n ∈ P (A) n , ∀i ∈ 1, n , range i = a i , a i + |x i | -1 , ∃i ∈ 1, n : range i ⊆ A ∨ range i ∩ supp (h) = ∅ ∨ range i ∩ i =j range j = ∅                        ∪ {(l, ρ) → ω | ρ ∈ M [V [l]] , i ∈ 1, n , ω ∈ π 2 ((e i ) (ρ))} ∪ τ P C [Body P (f )]
where x i are the names of the parameter of function f . It is interesting to remark that the functions that are not reachable from the entry point of the program are not translated, and thus generate no transition. It is similar to program linking where functions that are never referenced are not included in the final binary.

Initial States

To build the legal initial environments, we have to choose an entry point. The classical choice is to start by executing the function named main.

The stack of local environments contains a single empty call frame, the one of the main function. The stack of return address is empty. On the other hand, the environment for global variables is already full and the heap is initialized accordingly.

CHAPTER 4. A MINIMAL C-LIKE LANGUAGE

Formally

I =                                                (l, ρ) l ∈ L C , ρ ∈ E C : ∃v ∈ E C : ∃h ∈ H C : ∃g ∈ E C : ∃(a i ) i∈ 1,n ∈ A n : supp (v) = ∅, ∀(i, j) ∈ 1, n 2 , i = j ⇒ a i , a i + |t i | -1 ∩ a j , a j + |t j | -1 = ∅, supp (h) = n i=1 a i , a i + |t i | -1 , supp (g) = {x 1 , . . . , x n }, ∀i ∈ 1, n , g(x i ) = a i , ρ = ((v), ε, g, h) , l = at P (main)                                               
where t i x i ;, with i ∈ 1, n are the global definitions, where t i is a type and x i a variable name.

Comparison with Real C and Thoughts

Asclepius is clearly inspired by C, but have some differences. Some are significant, others are just a matter of syntactic sugar.

The Place of Undefined, Unspecified and Implementation-Defined Behaviors

There is a lot of cases in which C specifies that the behavior is undefined, that is anything can happen. This is definitely something to avoid. Implementation-defined and unspecified behaviors are less harmful: the standard lets some liberty to the compiler and the environment but it can still specify a set of allowed behaviors. The difference between both notions is that an implementation-defined behavior must be documented by the implementation (typically the compiler), while for an unspecified behavior, the implementation does not have to make a choice. For instance, the number of bits in a byte is implementation-defined, so the compiler must document its choice and stick with it. But the evaluation order of sub-expressions in an expression is unspecified (in most cases), so the compiler can choose any order and does not have to be consistent in any way, neither across several occurrences of identical expressions nor across several evaluations of the same expression. It is still correct for the compiler to specify an unspecified behavior (according to the standard), but we cannot expect it to do so. These kinds of behaviors are treated in very various ways in Asclepius. First, it is good to note that a lot of such problems in C come from modular programming (like external declaration) or from standard library functions. Since we do not have

COMPARISON WITH REAL C AND THOUGHTS

59

these features with Asclepius, we do not have to bother about that. Other cases are also out of our scope, like the possible data-race in sub-expression evaluation without sequence point † , since expressions of the minimal language are pure. Everything related to storage classes or qualifiers is also not applicable. Nevertheless, there is some place left for undefined behavior like dereferencing an illegal pointer or dividing by 0. The default behavior for Asclepius is to transit to an error state. Such situations can be detected dynamically since our abstract language is not subjected to the thousand natural shocks that software engineering is heir to: no architecture constraint, no efficiency problem. . . . For instance, it is possible to know which addresses are valid by looking which belong to the support of the heap. In practice, this is not possible. Thus, while a real-life dereference might be dangerous, in Asclepius, we can safely decide whether it is safe or not.

From the perspective of unspecified and implementation-defined behaviors, once we have ignored everything that is not applicable, what is left is largely defined in Asclepius. Some strong choices have been made here because of the application: to model interactions of C and assembly, we need a more precise description of some aspects of the language, way beyond the C standard. C is made to be (mostly) platform-independent, and to be efficiently achievable on a large variety of architectures, it needs to leave a lot of details unclear: these are the undefined, unspecified and implementation-defined behaviors. While working on mix of C and assembly, such details are very important, and we perfectly know on which architecture we are working on. We have at the same time both the necessity and the opportunity to define the language much more precisely.

One could argue that such extra hypotheses are not portable and that changing the architecture (or the compiler) may invalid most of the following. This is perfectly true, both conceptually and in practice. Indeed, in OSes, parts that interact with assembly or anything related to the architecture must exist in several versions, for each of supported architectures. The point is to make such parts as small as possible so that most code can be shared using regular platform-independent C or C++ (most of the time). Since we are interested into a specific architecture, we can make these assumptions. Moreover, without them, we would not be able to define a satisfactory semantics since we indeed use the specifics of the architecture.

Desugaring C

C has many features that do not exist in Asclepius, like for-loops, branching with else (and else-if) or switch statements. These features are very handy for the developer but can be easily avoided thanks to a simple desugaring.

C expressions are very rich and their semantics is very complicated. There are two main difficulties: side effects and types. C expressions may include assignments in a lot of places: they are regular expressions; this is not possible with Asclepius where expressions are pure and assignments are statements. C has some very complicated rules 60 CHAPTER 4. A MINIMAL C-LIKE LANGUAGE to deduce the type of expressions (and sub-expressions) based on the types of variables. This is not only a matter of bounds, since the semantics of operators changes depending on the types. For instance, an addition on unsigned integers is computed with modular arithmetic, but an overflow with signed integers is an undefined behavior. Asclepius dissimulates this problem under the chaste veil of n-ary operators in expressions. This set of operators is not fixed, we just need to know their semantics. In particular, we can interpret all implicit transformations in expressions (like integer conversions) in terms of such operators. In the same way, we can have operators for unsigned addition and others for signed addition for each type combination allowed by the C standard (after automatic conversions). Such transformation are performed by most tools working on C semantics, like compilers and analyzers, to clarify the expected behavior.

A missing feature in Asclepius, compared to C, is function pointers. This deserves some comments since we use them intensively with assembly. We can still emulate function pointers by using identifiers instead. So that, rather than writing p(x) where p is a function pointer, we could write call(p, x) where

int call(int id, int x) { if(id == 0) return f(x); if(id == 1) return g(x); // etc.

}

This is a bit long and not modular (since we need to be aware of all existing functions), but it has the expected behavior. Using a bit of sugar, we can make it the general case. In Asclepius' model, functions do not exist in the heap, so that, it is impossible to read or write their code, just as it is forbidden by C. By isolating functions from the memory, we do not let it happens. To add function pointers into Asclepius without compromising this isolation, a strategy is to have a global environment for functions: it is an injective partial mapping from addresses to functions. It is like having randomized identifiers. Calling a function by its address behaves like the call function: the semantics finds the requested function and calls it. Any function pointer that is not exactly the address of a function (or a function with a wrong type) is an immediate error.

A Computability Digression

Asclepius uses a finite memory: the heap is finite and environments are not usable as storage from the program's point of view. Moreover, since parameters are stored in the heap, even the recursion depth of a function with at least one parameter is limited. Only the number of calls of function without parameters are unlimited, since return addresses do not require any space. Such calls correspond to empty environments and can only read and write global variables, and thus have limited storage. Thus, we can decide whether such calls terminate in finite time since the number of configurations of global variables is an upper bound on the number of calls while still terminating.

Overall, due to memory limitation, Asclepius is not Turing-complete, but in the exact same way a computer with finite memory is not Turing-complete either. The amount of memory is fixed because of the application of this work (32-bit x86 architecture without external storage) and so it would mainly be a burden to make the available memory parametric, but could be done easily: we just need to change the size of addresses and extend the heap appropriately. Such parameters can be adapted to make any program run satisfactorily by providing an arbitrarily big memory: terminating programs will eventually be able to terminate without running out of memory and non-terminating programs can run for an arbitrarily long time (allowing, for instance, a non-terminating machine to generate as many decimal of a real number as wanted).

Beyond the concern of computable functions, we can wonder if the number of states is small enough to reasonably allow methods based on exhaustive state enumeration. In our case, it is clearly not reasonable. Moreover, if any restriction in Asclepius had made any of such method doable, it would not have been a good idea since it would not adapt well to an architecture with more memory or with external storage.

Non-determinism and Memory Usage

The semantics of Asclepius is non-deterministic in several ways. The most obvious one, and yet the most standard, is that expressions are non-deterministic. A more problematic point is memory allocation: when a variable is declared, it is placed wherever there is available space in the heap. This is a way a bit extreme to avoid strong assumptions on memory allocation policy; since there are many ways to manage memory, it is hard to accept only reasonable ones. But the good side is that what we will be able to prove with such weak hypotheses still hold with more. Making this few assumptions has two main consequences.

Firstly, since variables can be placed with any chaotic layout, we can fill 1 GiB using 2 30 1-byte variables spaced out by 3 bytes. If we try to allocate a 4-byte variable, allocation will fail since there is no hole big enough, though we use only a fourth of the memory. If the memory is tidy, we can put all the 2 30 1-byte variables at the beginning and still have an available contiguous block of 3 GiB, so that the next allocation will succeed. The moral of this example is that we should not attach too much importance to memory allocation errors. This may look like a limitation of this model but it is much wider. When we analyze some source code, we may consider that memory allocation functions (like malloc) can fail, but we rarely take into account that the declaration of a local variable may lead to a stack overflow and thus a crash of a perfectly legitimate program (from the point of view of C standard). To correctly take into account such kind of errors, we must go beyond C standard and know more precisely the memory management policy (are call frames very tight, or do we allocate 1 MiB of memory, even if we use only one integer?) and the available space. Such checks can only been meaningfully performed at a lower level. That is, for instance, what StackAnalyzer [START_REF] Sighireanu | AbsInt[END_REF] does. Another approach is to impose restrictions strong enough so that there is no memory layout messy enough to make allocations fail. For instance, we can prove that if we use a fraction smaller than 1 /n of the total memory (where n is the byte-size of the wider CHAPTER 4. A MINIMAL C-LIKE LANGUAGE type), no allocation can fail according to Asclepius' semantics. Such a hypothesis is very sensible for the following: most variables are global and recursion is forbidden, so memory usage of local variables is bounded. To sum up, we have strong hypotheses that encourage us not to worry about allocation errors, but if we really want to, it would need another analysis based on a more precise model. The second consequence of allocation at arbitrary addresses is that, perchance, a random integer can be a legal address. Let us consider the example shown on Listing 4.1. If x is allocated at address 42, x receives the value 1337. In any other situations, an illegal memory access happens and the program terminates in an error state. Thus, we need all executions to be valid, whatever the position of each variable in the heap, to declare the program valid. Some approaches avoid this problem by handling pointer more symbolically: by restraining how a pointer can be built and the valid operations on them (typically sum and difference). With this approach, pointers are characterized by the pointed variable and an offset. The non-deterministic part (the address of the pointed variable) is abstracted away. A major downside is that it does not allow arbitrary computations on pointers, like C does. In C, it is possible to convert integer into pointers and conversely (even if the behavior is implementation-defined) and low-level source codes make extensive use of such conversions and perform complex (arithmetic and bitwise) computations on pointers. To represent faithfully such computations, we need to treat pointers like integers, even if it means to pay the price of additional nondeterminism.

Chapter 5

The Assembly Language I ntroducing Asclepius as a model of C is the first part toward the modeling of mixed C and assembly. It is now time to study the assembly language of x86 processors. It is very different from a C-like language: it has no structures, there is no arbitrarily complex expressions and no compound statements. Most instructions are quite simple, but they are many of them. Moreover, the notion of undefined behavior is very different: it is more similar to unspecified behavior of C. Implementation-defined behavior is not a relevant notion since we are already at the lower level * . For all these differences, our approach of x86 is not the same as for C. We do not need a more precise model, but we are interested only in a fragment of the language since there are too many instructions to treat them all.

An interesting, and almost exhaustive, work on the semantics of x86 assembly can be found in [START_REF] Dasgupta | A Complete Formal Semantics of x86-64 User-Level Instruction Set Architecture[END_REF]. Unfortunately, it is not very useful here. First, it focuses only on user-level instructions and, while we still need to handle some of them, we are mostly interested into system-level instructions. Though a formalization could have been useful to solve ambiguities of [Int20] † , we mostly needed it about instructions out of the scope of [START_REF] Dasgupta | A Complete Formal Semantics of x86-64 User-Level Instruction Set Architecture[END_REF]. Moreover, it does not take mixed C and assembly into account, which is our main problem. Finally, in [START_REF] Dasgupta | A Complete Formal Semantics of x86-64 User-Level Instruction Set Architecture[END_REF], everything is made in the K framework, which is not at all our context. We give two semantics for assembly. The first one is very low-level: it sees programs as sequences of bytes in memory. This is a very permissive semantics: we can write everywhere, including in the code, and run data. Actually, there is no formal distinction 64 CHAPTER 5. THE ASSEMBLY LANGUAGE between code and data. This semantics is interesting as it is very close to the processor's behavior. Moreover, it is quite simple and some parts are common with the second semantics. It is simpler to define the semantics of some instructions according to the first semantics, and then to make the few adaptations needed for the second one.

The second semantics models code as a sequence of statements that are in memory, but with an unknown layout. This part of memory is not writable, so that the sequence of statements is immutable. However, data (the writable part of memory) is still executable, since this is something that exists in our study case.

Structure of x86 Assembly Language

As far as we are concerned, assembly code contains instructions, labels and pseudoinstructions (or directives). Instructions are translated by the assembler but the binary encoding is almost equivalent. Labels are used to mark points in the source code. They are removed at assemble-time and replaced by their respective address (or by the offset, if applicable). Pseudo-instructions are instructions executed by the assembler. They allow manually writing bytes in the binary, and they provide a preprocessing mechanism similar to C's preprocessor. We will only use directives that place bytes manually where these directives are written.

Instructions are identified by a keyword (called mnemonic ‡ ), followed by commaseparated arguments. Once assembled, the instruction is identified by a binary sequence of 1 or 2 bytes. Different mnemonics can be aliases for the same opcode. Depending on the opcode, there can be 0 to 2 arguments. Labels are identifiers followed by a colon. Directives are the responsibility of the assembler, the syntax is more fluctuating. They may look like instructions whose opcode is not an x86 instruction, or can be prefixed with a single dot. Examples are given on Listing 5.1. Comments begin with a semicolon and stop at the end of the line. Keywords are not case-sensitive.

Instructions may be ambiguous (or at least, hard to fully determine) in various way: size of the assembled instruction, operand size, ambiguous mnemonics. . . . Such ambiguities exist only in assembly language, but not in machine language (binary encoding): they are solved by the assembler. Consequently, we assume for now that instructions are fully explicit, even if it means that we add some annotations. The resolution of ambiguity will be treated later in chapter 9 "Amical" (page 121).

They are three kinds of operands: register, memory and immediate. Register operands designate a register by its name. Memory operands are dereference, optionally with an explicit segment selection. Immediate operands are constant values, corresponding to literal constants of most languages. In practice, some operands may be written as a name, e.g. immediate operands designating an address may be expressed as a label or a memory operand designating a global variable may be expressed as the variable's name.

Memory operands, when not expressed as a variable name, allow some level of complexity in addresses. Allowed address forms are shown on Figure 5

            EAX EBX ECX EDX ESP EBP ESI EDI             Base +                          EAX EBX ECX EDX EBP ESI EDI           Index ×     1 2 4 8     Scale                +     None 8-bit 16-bit 32-bit     Displacement Figure 5
.1: Allowed memory addresses is a literal offset to add. For instance, in the address EDX + EDI * 4 + 0x0ff537, EDX is the base, EDI is the index, 4 is the scale and 0ff537H is the displacement. In this example, the displacement is 20-bit large and thus, will only fit in a 32-bit displacement.

All of these fields are optional and, in most cases, they are not all used at the same time. The scale provides a nice way to iterate over arrays with 2-, 4-or 8-bytes cells. If the scale is 1, it can be implicit. The base have a special role since it is the one used to choose the segment in which the memory operand should be translated, except when it is explicitly specified.

Memory Model for Assembly

In assembly, the memory is view through segments and paging (see section 2.3 "Memory Management" (page 17)). We will abstract paging away since it is not always enabled and linear address space is already concrete enough for our concern. We do not need to CHAPTER 5. THE ASSEMBLY LANGUAGE know how linear address space is mapped over physical memory as long as it behaves like a 4 Gio memory block.

On the other hand we must take segmentation into account, in some extent. Indeed, we have already explained that it is common to have segments that alias each other (see subsubsubsection 2.3.2.2.2 "Segmenting the Linear Address Space" (page 21)). But to know the effect that an assignment has in other segments, we must know precisely how they alias. So, we will only consider the case where all segments start at address 0 and have the same size. This means we should check that this is true when we load a segment register. On the good side, this is a powerful assumption that allows us to almost forget about segments. Segmentation being trivial, working with linear addresses is almost the same as working with physical addresses.

Another touchy point is about memory allocation. It is very different depending on the context. In the system world, at boot, all the memory is its kingdom. Nevertheless, the system must be very careful not to use memory that will be allocated for ordinary programs later. For a user land program, it is the opposite: any memory access that have not been explicitly granted is strictly forbidden. Each piece of memory must be allocated either at program launch (for static and global variables) by the loader, or dynamically (for dynamically allocated blocks) by calling functions such as malloc.

In our context, we will avoid allocation questions. We are interested in an embedded OS: it does not perform dynamic allocation for itself, all long-lived structures are global variables. The only dynamic structure is the stack though it has bounded size. We can model it as a static block of memory at the end of which ESP initially points to. It is very similar in Asclepius: the only way to allocate a non-constant amount of memory is thanks to local variables.

In addition to the linear memory, we need to add registers into account. We have to be able to use registers as lvalue while they are not accessible through a pointer. To do that while being as close as possible to the formalism of Asclepius, we enrich the domain of the heap with special values that are not integers, and we forbid to take the address of an lvalue located in such special zone. Moreover, we add special mappings in the global environment for registers. More formally, let A R be a set of identifiers for each register byte such that A ∩ A R = ∅. We denote such identifier by the register names, excluding alternate names for shorter parts (e.g. AX, see subsubsection 2.2.1.1 "General-Purpose Registers" (page 10)), with the byte number. That is

A R :=            EAX 0 , EAX 1 , EAX 2 , EAX 3 , EBX 0 , EBX 1 , EBX 2 , EBX 3 ECX 0 , ECX 1 , ECX 2 , ECX 3 , EDX 0 , EDX 1 , EDX 2 , EDX 3 ESI 0 , ESI 1 , ESI 2 , ESI 3 , EDI 0 , EDI 1 , EDI 2 , EDI 3 ESP 0 , ESP 1 , ESP 2 , ESP 3 , EBP 0 , EBP 1 , EBP 2 , EBP 3 . . .           
We also need a set of register names R, including alternate names i.e.

R :=        EAX AX AL AH ESP EBP SP BP ESI EDI SI DI . . .       
We define the set of enriched heaps:

H asm := {f : (A A R ) I | A R ⊆ supp (f )}
and we add a global constant mapping

reg : R → A R EAX → EAX 0 AX → EAX 0 AL → EAX 0 AH → EAX 1 . . .
| • | : R → N * EAX → 4 AX → 2 AL → 1 AH → 1 . . .
On the other hand the environment has the same type as it maps only global variables to their address:

E asm := E C = V A
The memory model is then very close to the one of Asclepius, except that local variables are not relevant here and do not exist in assembly. Thus, we just need an environment for global variables and a heap. Like global variables, the stack has an arbitrary position in the heap at the beginning of the program.

There is the subtle problem of the place of the program in memory. We can indeed take the address of some program point (usually thanks to a label) and use them in control flow instructions. It may be possible to read code as data, depending on code segment permission flags. Moreover, we can write code through the data segment and run it through the code segment. A simple strategy is to forbid it, but even if these manipulations are risky, but they exist, and we must take them into account.

The idea is similar to the allocation of global variables: we choose an arbitrary position in memory for the source code, and we keep it. Thus, we have to represent the program as bytes and to be able to disassemble those bytes to get instructions. For this purpose, we let Stat asm the set of assembly statements, and we define the function disassemble : H asm → (Stat asm × A) Ω that takes a heap (with register), disassembles the instruction at the address in EIP and yields this instruction with the address of the next instruction. We need the address of the next instruction since they have non-constant size in x86. In case of error, an element of Ω is returned instead: ω a if an attempt to access a non-reserved memory zone is made, and ω i (illegal instruction) if the byte sequence at the given address is not a legal instruction.

To start correctly the program, we have to make a few assumptions: initially, -ESP points to the byte after the end of the stack, -EIP points to the first program instruction. In addition, we need hypotheses on the state of the system, especially if protection features are used. For instance, if PE (protection enable) flag (bit 0 of CR0) is set:

-GDTR points to a correct GDT, -CS selects a code segment, -other segment registers select a data segment.

The Reduced Instruction Set

The complete instruction set of x86 is huge, even for the relatively old processor we are studying. There are several reasons for that:

-there are a lot of mnemonics providing a very broad set of operations including complex floating-point, binary-coded decimal (BCD) and other complex computations, -a lot of mnemonics have several variants for various types of arguments, -the complexity of x86 architecture requires many instructions to control it. We are not so concerned about the types of arguments as it is mainly an encoding concern, and we are not interested in for now. The multiplicity of instructions is a problem, but there are big categories. There are many instructions that take a lvalue and an operand, perform a binary operation (like the addition or the bitwise and) and write the result in the lvalue. This family includes a lot of arithmetic or logic operations, sometimes unexpected. They are also very close to the MOV instruction that copies a value into an lvalue. Another category often distinguished is the set of control flow operations, like jumps, calls and returns.

Technically, control flow instructions might be understood as compound instructions made of MOV-like instructions, since a near jump is just an assignment in EIP. The only MOV instruction is surprisingly powerful as it is Turing-complete by itself [START_REF] Dolan | mov is Turing-complete[END_REF]Dom15]. Control flow operations do not have a particular status like they have in most languages. However, they are good didactic examples and thus are worth to be detailed.

On the other hand, the semantics of a lot of complex instructions can be expressed as a composition of simpler transfer functions; they will not be detailed further.

We can thus drastically restrict the instruction set. We will use:

-MOV, as it is very fundamental, -ADD, to give an example of binary operation that both reads and writes a lvalue, -JMP, CALL, RET, to show basic control flow instructions that will be extended when interacting with C code, -JZ, to illustrate conditional control flow, -PUSH and POP, as stack operations are very common and good examples. From there, we trust that we can easily generalize to other instructions.

A Very Concrete Semantics

In the next chapters we will want to glue semantics of C (approximated by Asclepius) and assembly. For this purpose, it is wise to choose the same formalism, so the semantics of an assembly program is given as a transition system as well (see definition 4.6 "Transition system" (page 49)).

These transition systems have a major difference with the ones for Asclepius programs: the set of states is very different: the memory model is enriched with registers, and we have no label to get the program point. Indeed, it is stored in the EIP register as the address of the next instruction to run.

In this semantics, since we are considering machine code instead of assembly language, there is no name, so no need for an environment.

M asm := H asm and thus, the set of state simply is

Q := M asm Ω
Another fundamental difference between C and assembly is that, in assembly the program is in the memory, not external. Thus, transitions between states do not depend on an external current statement since the current statement is in the memory. Consequently, there is not a transition system for each program, but a single, huge transition system between memory states. This semantics is the most concrete possible. We will define another one later that is slightly more constraint as it rejects some unwanted behaviors.

There are two kinds of lvalues (registers and memory references) and three kinds of operand (registers, memory references and immediate operands).

Definition 5.1 -Dereferencing

Let h ∈ H asm .

deref h : P (A A R ) × N * → P (Z) × P (Ω) (A, size) → a∈A π 1 (d) , a∈A π 2 (d) where d = deref h (a, size)
where

deref h : (A A R ) × N * → P (Z) × P (Ω) (a, size) →                        (∅, {ω a }) if a ∈ A ∧ a, a + size -1 ⊆ supp (h) size-1 i=0 h(a + i) • 2 8i , ∅ if a ∈ A ∧ a, a + size -1 ⊆ supp (h) (∅, {ω a }) (absurd) if a = R n ∈ A R ∧ ∃i ∈ 0, size -1 : R n+i ∈ A R size-1 i=0 R n+i • 2 8i , ∅ if a = R n ∈ A R ∧ ∀i ∈ 0, size -1 : R n+i ∈ A R
where ω a is the error raised by an illegal memory access.

The case marked as impossible is it since a register name is always valid, but this case is specified for exhaustiveness.

It may seem a bit restrictive to return an error in case of an illegal memory access since in reality it will raise an exception that can be handled. Actually, the error can be captured and treated in any way by the semantics of instructions. In practice, we do not want such accesses and it is safe to consider them as errors, but it is interesting to remark that returning an error is not an intrinsic requirement.

Definition 5.2 -Semantics of lvalues and expressions

Semantics of lvalues and expressions are defined by mutual recursion. The semantics of an lvalue l is

⦉l⦊ : M asm → P (A A R ) × P (Ω) h →                             (b + scale × i + v) mod 2 32 , O b ∪ O i ∪ O v where (b, O b ) = (base) (h) and (i, O i ) = (index) (h) and (v, O v ) = (imm) (h) if l = [base+scale*index+imm] (reg (l) , ∅) if l ∈ R
and the semantics of an expression e is (e) : M asm → P (Z) × P (Ω)

h →          e mod 2 8•|e| , ∅ if e ∈ Z    V, O ∪ O where (V, O ) = deref h (A, |e|) where (A, O) = ⦉e⦊ (m) if e = lvalue
Now, let build the transition system. We assume we are given a memory space for the stack called stack. It is a set of consecutive cells in memory.

Let h ∈ M asm a memory state. Let d = disassemble (h). If d ∈ Ω, then d is the only successor of h.

If d = (stat, new_eip) ∈ Stat asm × A, we let h = h [EIP → new_eip] ∈ M asm .
We perform a case analysis on stat to deduce the set of states we can reach from h.

-stat = MOV l, r:

       h ∈ M asm ∃v ∈ Z : ∃a ∈ A : v ∈ π 1 ((r) (h)) , a ∈ π 1 ⦉l⦊ (h) : h = h [a + i → v [8i : 8i + 7]] |l|-1 i=0        ∪        h ∈ M asm ∃v ∈ Z : ∃R n ∈ A R : v ∈ π 1 ((r) (h)) , R n ∈ π 1 ⦉l⦊ (h) h = h [R n+i → v [8i : 8i + 7]] |l|-1 i=0        ∪ ω ∈ Ω ω ∈ π 2 (r) h ∪ ω ∈ Ω ω ∈ π 2 ⦉l⦊ h
The bytes of r are copied in l.

CHAPTER 5. THE ASSEMBLY LANGUAGE

-stat = ADD l, r:

                                 h 2 ∈ M asm ∃(v 1 , v 2 , ef lags) ∈ Z 2 : ∃v ∈ 0, 2 32 -1 : ∃a ∈ A : ∃h 1 ∈ M asm : v 1 ∈ π 1 ((l) (h)) , v 2 ∈ π 1 ((r) (h)) a ∈ π 1 ⦉l⦊ (h) v = (v 1 + v 2 ) mod 2 32 h 1 = h [a + i → v [8i : 8i + 7]] |l|-1 i=0 ef lags = π 1 ((EFLAGS) (h 1 )) h 2 = h 1 [EFLAGS i → update_eflags(v 1 + v 2 , ef lags) [8i : 8i + 7]] 3 i=0                                  ∪                                  h 2 ∈ M asm ∃(v 1 , v 2 , ef lags) ∈ Z 3 : ∃v ∈ 0, 2 32 -1 : ∃R n ∈ A R : ∃h 1 ∈ M asm : v 1 ∈ π 1 ((l) (h)) , v 2 ∈ π 1 ((r) (h)) R n ∈ π 1 ⦉l⦊ (h) , v = (v 1 + v 2 ) mod 2 32 h 1 = h [R n+i → v [8i : 8i + 7]] |l|-1 i=0 ef lags = π 1 ((EFLAGS) (h 1 )) h 2 = h 1 [EFLAGS i → update_eflags(v 1 + v 2 , ef lags) [8i : 8i + 7]] 3 i=0                                  ∪ ω ∈ Ω ω ∈ π 2 (r) h ∪ ω ∈ Ω ω ∈ π 2 (l) h ∪ ω ∈ Ω ω ∈ π 2 ⦉l⦊ h where update_eflags : Z × 0, 2 32 -1 → 0, 2 32 -1 (v, e) → c + p + a + z + s + o + i∈{1,3,5} 2 i • e [i : i] + 2
where c = 2 0 • v < 0 ∨ v 2 32 and p = 2 1 • 7 i=0 v [i : i] ≡ 0 [8] and a = 2 4 • [A ] and z = 2 6 • v ≡ 0 2 32 and s = 2 7 • v [31 : 31] and o = 2 11 • v < -2 31 ∨ v 2 31

A VERY CONCRETE SEMANTICS

where [P ] is Iverson's bracket (see section A.2 "Logic" (page 370)) and A is the condition for AF flag § . Bytes of the result of the addition are written into l.

-stat = JMP m where m is a memory operand:

     h ∈ M asm ∃a ∈ A : a ∈ π 1 (m) h h = h EIP i → a mod 2 32 [8i : 8i + 7] 3 i=0      ∪ ω ∈ Ω ω ∈ π 2 (m) h
EIP is updated to the value contained in m.

-stat = JMP o where o is an immediate offset:

               h ∈ M asm ∃(a, eip) ∈ Z 2 : ∃d ∈ A : a ∈ π 1 (o) h eip ∈ π 1 (EIP) h d = (a + eip) mod 2 32 h = h [EIP i → d [8i : 8i + 7]] 3 i=0                ∪ ω ∈ Ω ω ∈ π 2 (o) h ∪ ω ∈ Ω ω ∈ π 2 (EIP) h
EIP is incremented by of o. It is worth noticing that the EIP we increment is already the address of the next instruction. § It is too difficult to formalize here, especially for a flag we do not use. But such a condition exists, that's all we need. Since we do not use this flag ¶ , we can simply exclude it from the model.

¶ It is relevant only for BCD computations.

CHAPTER 5. THE ASSEMBLY LANGUAGE

-stat = CALL m where m is a memory operand:

                                       h 3 ∈ M asm ∃v ∈ Z : ∃a ∈ A : ∃(h 1 , h 2 ) ∈ M asm 2 : ∃(esp, eip) ∈ Z 2 : v ∈ π 1 (m) h a = v mod 2 32 h 1 = h [EIP i → a [8i : 8i + 7]] 3 i=0 esp ∈ π 1 (ESP) h eip ∈ π 1 (EIP) h (esp -1) mod 2 32 , (esp -4) mod 2 32 ⊆ stack h 2 = h 1 ESP i → ((esp -4) mod 2 32 ) [8i : 8i + 7] 3 i=0 h 3 = h 2 esp -i -1 → (eip mod 2 32 ) [8i : 8i + 7] 3 i=0                                        ∪      ω a ∈ Ω ∃esp ∈ Z : esp ∈ π 1 (ESP) h (esp -1) mod 2 32 , (esp -4) mod 2 32 ⊆ stack      ∪ ω ∈ Ω ω ∈ π 2 (m) h ∪ ω ∈ Ω ω ∈ π 2 (ESP) h ∪ ω ∈ Ω ω ∈ π 2 (EIP) h
It is similar to JMP m but EIP is pushed on the stack.

A VERY CONCRETE SEMANTICS

75

-stat = CALL o where o is an immediate offset:

                                             h 3 ∈ M asm ∃a ∈ Z : ∃a ∈ A : ∃(h 1 , h 2 ) ∈ M asm 2 : ∃(esp, eip) ∈ Z 2 : a ∈ π 1 (o) h a = a mod 2 32 esp ∈ π 1 (ESP) h eip ∈ π 1 (EIP) h (esp -1) mod 2 32 , (esp -4) mod 2 32 ⊆ stack d = (a + eip) mod 2 32 h 1 = h [EIP i → d [8i : 8i + 7]] 3 i=0 h 2 = h 1 ESP i → ((esp -4) mod 2 32 ) [8i : 8i + 7] 3 i=0 h 3 = h 2 esp + i → (eip mod 2 32 ) [8i : 8i + 7] 3 i=0                                              ∪      ω a ∈ Ω ∃esp ∈ Z 2 : esp ∈ π 1 (ESP) h (esp -1) mod 2 32 , (esp -4) mod 2 32 ⊆ stack      ∪ ω ∈ Ω ω ∈ π 2 (o) h ∪ ω ∈ Ω ω ∈ π 2 (ESP) h ∪ ω ∈ Ω ω ∈ π 2 (EIP) h
It is similar to JMP o but EIP is pushed on the stack. -stat = RET:

                           h 2 ∈ M asm ∃v ∈ Z : ∃(esp, eip) ∈ Z 2 : ∃h 1 ∈ M asm : v ∈ π 1 (([ESP]) (h)) esp ∈ π 1 (ESP) h eip ∈ π 1 (EIP) h esp mod 2 32 , (esp + 3) mod 2 32 ⊆ stack h 1 = h EIP i → v mod 2 32 [8i : 8i + 7] 3 i=0 h 2 = h 1 ESP i → ((esp + 4) mod 2 32 ) [8i : 8i + 7] 3 i=0                            ∪      ω a ∈ Ω ∃esp ∈ Z : esp ∈ π 1 (ESP) h esp mod 2 32 , (esp + 3) mod 2 32 ⊆ stack      ∪ {ω ∈ Ω | ω ∈ π 2 (([ESP]) (h))} ∪ ω ∈ Ω ω ∈ π 2 (ESP) h ∪ ω ∈ Ω ω ∈ π 2 (EIP) h
This instruction pops the top of the stack and stores it in EIP.

-stat = JZ o where o is an immediate offset:

                     h ∈ M asm ∃(a, eip) ∈ Z 2 : ∃d ∈ A h (EFLAGS 0 ) [6 : 6] = 1 a ∈ π 1 (o) h eip ∈ π 1 (EIP) h d = (a + eip) mod 2 32 h = h [EIP i → d [8i : 8i + 7]] 3 i=0                      ∪ h ∈ M asm h (EFLAGS 0 ) [6 : 6] = 0 ∪ ω ∈ Ω ω ∈ π 2 (o) h ∪ ω ∈ Ω ω ∈ π 2 (EIP) h
This instruction performs a relative jump if ZF flag is set. Otherwise, it does nothing.

-stat = PUSH r:

                       h 2 ∈ M asm ∃(v, esp) ∈ Z 2 : ∃h 1 ∈ M asm : v ∈ π 1 (m) h esp ∈ π 1 (ESP) h (esp -1) mod 2 32 , (esp -|r|) mod 2 32 ⊆ stack h 1 = h ESP → ((esp -|r|) mod 2 32 ) [8i : 8i + 7] 3 i=0 h 2 = h 1 esp + 8i → (v mod 2 32 ) [8i : 8i + 7] 3 i=0                        ∪      ω a ∈ Ω ∃esp ∈ Z : esp ∈ π 1 (ESP) h (esp -1) mod 2 32 , (esp -|r|) mod 2 32 ⊆ stack      ∪ ω ∈ Ω ω ∈ π 2 (m) h ∪ ω ∈ Ω ω ∈ π 2 (ESP) h
It pushes the value of r on the stack.

A MORE SYMBOLIC SEMANTICS

77

-stat = POP l: It pops the top of the stack and stores it l.

                       h 2 ∈ M asm ∃(v, esp) ∈ Z 2 : ∃a ∈ A : ∃h 1 ∈ M asm : v ∈ π 1 (([ESP]) (h)) , a ∈ π 1 ⦉l⦊ (h) esp ∈ π 1 (ESP) h esp mod 2 32 , (esp + |l| -1) mod 2 32 ⊆ stack h 1 = h a + i → v mod 2 32 [8i : 8i + 7] |l|-1 i=0 h 2 = h 1 ESP i → ((esp + |l|) mod 2 32 ) [8i : 8i + 7] 3 i=0                        ∪                        h 2 ∈ M asm ∃(v, esp) ∈ Z 2 : ∃R n ∈ A R : ∃h 1 ∈ M asm : v ∈ π 1 (([ESP]) (h)) , R n ∈ π 1 ⦉l⦊ (h) esp ∈ π 1 (ESP) h esp mod 2 32 , (esp + |l| -1) mod 2 32 ⊆ stack h 1 = h R n+i → v mod 2 32 [8i : 8i + 7] |l|-1 i=0 h 2 = h 1 ESP i → ((esp + |l|) mod 2 32 ) [8i : 8i + 7] 3 i=0                        ∪      ω a ∈ Ω ∃esp ∈ Z : esp ∈ π 1 (ESP) h esp mod 2 32 , (esp + |l| -1) mod 2 32 ⊆ stack      ∪ {ω ∈ Ω | ω ∈ π 2 (([ESP]) (h))} ∪ ω ∈ Ω ω ∈ π 2 (ESP) h ∪ ω ∈ Ω ω ∈ π 2 ⦉l⦊ h
This is the transition relation. We need the initial states. Informally, they are the states were global variables, stack and code are disjoint. ESP points at the end of the stack, and EIP points in the code.

A More Symbolic Semantics

The previous semantics is almost what the processor does. It is interesting because it is quite simple and it helps to describe what we can expect from a higher-level point of view: assembly language source code. Though assembly and machine code are very close they have slightly different constraints that we have to take correctly into account.

We have global variables designated by their names and program points are pointed by labels. Moreover, we do not have access to the machine code, but only the assembly source code. Since the assembling is not unique and a single assembly instruction may have machine encoding of different length, this has some consequences on control flow operations: the destination of a jump whose destination is dynamic is not precisely known. Moreover, if a jump leads to the middle of an instruction, we consider it as an error. Indeed, such jumps are dubious, for the least, and since we do not know the CHAPTER 5. THE ASSEMBLY LANGUAGE machine code, we do not know what instructions we would get by reading them with some offset * * .

At this level, a program is a sequence of statements. Since it has a flat structure, we can use integers to specify a program point: the index of the statement following the program point. There is no need for complicated labels like in Asclepius. Moreover, labels (in the assembly meaning) are mere aliases for program points. We need to assume that the code lies in memory so instructions have conventional addresses, but we do not want the code to be writable, since the assembly statements are seen as immutable. Nevertheless, we cannot completely get rid of dynamic code, so we need to allow data to be run as code. It is not so often useful, but since it happens in the study case, we need to model it. Executing assembly source code is like executing a single sequence. Running dynamic code (byte sequence in a variable, typically an array) is the same as previously: we disassemble bytes to get corresponding instructions. However, the world of assembly source code is not entirely separated from the data world: each program point correspond to an address in memory. It is similar to the variables in Asclepius; they can be accessed through their names or by a pointer which may be the result of complicated computations. For instance, Listing 5.2 shows an example where we perform computations on labels but still get a valid destination anyway.

It is not always straightforward to transform an address to a program point for two reasons: the position of the source code is not statically know, and a statement may be assembled in multiple byte sequences with unequal lengths. For instance, the statement add EAX, 3 can be assembled in three ways:

-0x05 0x03 0x00 0x00 0x00: using the opcode that adds a 32-bit immediate operand to EAX; -0x81 0xc0 0x03 0x00 0x00 0x00: using the opcode that adds a 32-bit immediate operand to a register or memory operand; -0x83 0xc0 0x03: using the opcode that adds a 8-bit immediate operand to a register or memory operand. These sequences have respectively size 5, 6 and 3. Thus, the code in Listing 5.3 is valid only if the assembler choose the 3-bit assembling so, in general, this code is invalid since we cannot guarantee the behavior of the assembler. In fact, most assemblers claim to choose the shortest assembling possible, but we have tested several common assemblers, and they all failed at least once to do so. There are some good reasons to choose another sequence, but the point is that we cannot rely on this assumption. This is slightly more developed in section 9.2 "Assembling" (page 125). To take this difficulty into account, at the beginning, we choose the length of each statement and the position of the code, just like we choose the position of the stack.

We will not give explicit formal rules like before because it does not make the semantics more understandable. Nevertheless, it is important to detail the type of the set of states of the transition system.

States must keep the program point. This is stored indirectly: the current code pointer is stored in EIP. Moreover, in the initial state, we choose the size of each instruction and the position of the code in memory. This allows deciding whether the code pointer is a valid program point and, in this case, to get it. So, in addition of the heap, we need the base address of the source code and a map of type statement_size : 0, n -1 → N * where n is the number of statements, such that statement_size (i) is the size of the i th statement. The base address should be such that the last byte of the last statement is still in A. Function statement_size belongs to the state since it is not intrinsic to the program (since there are several possible mapping), but it is constant along an execution trace. This map is chosen in the initial states.

Overall, the memory model for this adapted semantics is

M asm := H asm × A × N * 0,n-1
The first component is the content of the memory strictly speaking. The second is the address of the first instruction in memory. The third component is the size of each instruction, so that we known the position of each instruction in memory. As said before, only the first component can change along an execution trace, the two others are immutable.

Comparison of Semantics

The first semantics is very close to the way the processor indeed runs the program. It assumes we have chosen a place in memory for the program (for which we have only the binary encoding) and the stack. This transition system is a bit heavy to write, but conceptually simple. Though it is not suited to the formalization of interactions with C. In C (like most programming language), we abstract the actual layout of memory (we see it only through variables) and the code is considered immutable. The second semantics is very similar up to slight adaptations that aim to solve these problems. The goal is to make it as close as possible of the version we will have in mixed code, while still making sense in pure assembly code. In this semantics, the code is considered as an immutable sequence of instructions in a known region of memory, but with unknown binary content. There are several implications, among them, we can highlight that -there is no need to disassemble regular code (only when executing data); -writing into code is illegal and results into error state ω a (invalid address). Other changes include using global variables designated by their name as an alias for a memory location (with non-deterministic position in code), and labels in assembly code.

We do not allow the program to modify its own source code, since it is considered buggy, and we should not be able to do it anyway, thanks to memory protection features. But we cannot only rely on them for two reasons. Firstly, such features must be configured and enabled, and before that, there is no such protection, but we would like not to write the code anyway. Moreover, even if such protections are activated, a program that tries to modify a read-only memory (like its code) will be terminated, which is not a desirable behavior. Protections features are still useful for untrusted programs, so that we can ensure they will not modify its own code. For instance, this is useful to avoid dangerous instructions: if they are not in the executable generated by the compiler and the program is unable to modify its code, they will never appear later † † .

Overall, to build the second semantics, we have applied very light changes, but make this model of assembly much closer to what we will find mixed with C. Remaining problems are only about interaction, like operand sharing and control flow.

Chapter 6

Non-semantic Aspects of Mixed C and Assembly W e have already examined C and assembly separately, we are now interested into mixing them. Such combination of languages is not straightforward at all and problems appear even at syntactic level. This chapter is dedicated to the description of mixed C and assembly code from a non-semantic point of view. Semantics questions will be considered later.

Syntax of Assembly Blocks

There is a widespread GNU extension that defines a syntax to add assembly blocks in C source code. Sadly, this is not the syntax used by the study case. In our syntax, assembly blocks are opened by "asm {" and closed by "}". Such blocks can only appear in C functions, not as top-level declarations * . An example of such code is given in Listing 6.1.

1 int c_function() { 2 // C code 3 asm { 4 ; x86 code 5 } 6 // C code 7 }
Listing 6.1: Generic example of mixed C and assembly * In some variation of this syntax, blocks can be top-level items, especially to define functions. In our case assembly functions are always wrapped into a dummy C function. We will show an example in Listing 6.8 of section 6.3 "Sharing Symbols" (page 85). Supporting top-level assembly blocks would imply a minor modification of the parser, and a relatively light update of the following.

CHAPTER 6. NON-SEMANTIC ASPECTS OF MIXED C AND ASSEMBLY

GNU-style inline assembly requires specifying more than just the assembly code. There are also fields to list the read expressions, written lvalues and modified registers. There are no such things in this syntax. Operands will be detailed in the next section, but they do not need to be given explicitly to the assembly block. This is quite handy for the user but it has an obvious drawback. Either the compiler must parse assembly blocks to infer which register are modified, which variables may be written etc., or it has to compile just as if every assembly block can read and write everything, limiting greatly the optimization potential.

Sharing Operands

To allow interaction of C and assembly, they must share variables. Registers are not visible from C, consequently it should go the other way: C variables are used in assembly. The way a variable may be used in assembly varies according to its storage class.

Global Variables

The simplest case is the one of global variables. Referencing a global variable is a memory operand whose content is the variable itself. Indeed, global variables are statically allocated into the heap, so naming the global variable is just like dereferencing the address at which the compiler stored it. It is the same in assembly, and the dereferencing is also implicit. Consequently, in Listing 6.2, naming x in the assembly code is like writing [foo] where foo = &x. The assembly block in this example is equivalent to the simple C statement x = 4;.

Since x86 statements can have at most one memory operand, the example of Listing 6.3 is illegal since the MOV instruction has two memory operands: [EAX] (the value pointed by EAX) and x (a global variable obtained by dereferencing its static address). When we try to assemble such an instruction, GNU Assembler 2.30 emits the error message "too many memory references for mov".

Register Storage Class

In C, variables can be declared with the register storage class, thanks to the eponymous keyword. This keyword declares an automatic variable † , but also hints the compiler to store this variable into a register, mainly for performance purpose. This keyword is deprecated in C++, and even unused since C++17. Indeed, compilers are incredibly efficient nowadays and it is almost always better to let them make this kind of choices. In C, its meaning slightly changed: since registers cannot be reached through dereferencing, declaring a variable with the register storage class forbids to take its address (C's unary operator &), which can help alias analysis.

However, in our case, this keyword is important: it is not just a hint, but it enforces that the variable is indeed stored in a register. The point is not a performance issue, but that we can use this variable as a register operand in assembly instructions. In particular, it can be used with opcode that can only take a register as parameter (like MOV from or to a control register) and be used in a statement whose other argument is a memory operand. For instance, Listing 6.4 is valid since x is a register operand here.

Static Local Variables

Static local variables are local only from the perspective of scopes (where the name of the variable exists), but from the function where it is defined, a static local variable seems global (static storage duration: the lifetime of a static local variable is unbounded).

Indeed, for the value to be preserved, it is stored on the heap like a global variable, but the compiler takes care to check it is only accessed by its name in the function it is declared in, though it can still be accessed through a pointer.

Thus, using a static local variable in assembly is like using a global variable: it is a memory operand and it is used simply by its name. Listing 6.5 shows such an example.

Regular Local Variables

Regular local variables are much weirder. They are stored in the stack thus they are a memory operand, but their addresses are not static since it depends on the size (and the position) of the stack, which adds a difficulty. Let us make an aside on compilation. When a function is called in C, the stack contains the parameters, the return address, and the previous copies of ESP and EBP, all of them according to a layout defined by the calling convention. Later, in section 6.5 "Calling Convention" (page 87), we will see more about calling conventions. On the top of this (so, at lower addresses), there are local variables. Usually (but this is up to the compiler), EBP is used to point the first local variable, while ESP still points to the top of the stack. Thus, all local variables are stored between EBP and ESP. The top

1 void f() { 2 int x; 3 asm { 4 mov x[EBP], 4 ; Stores 4 in x 5 } 6 }
Listing 6.7: Assigning a C local variable from assembly of the stack may vary. For instance in the code on Listing 6.6, at label l0, there are two living local variable (x and y), while at label l1, x is the only living local variable. In this kind of situation, the compiler can act in various way. It can only allocate x at the beginning of the function, and then decrease ESP to allocate y on demand. With this strategy, if we go by the "else" branch, no additional space is allocated. This is very frugal way to handle memory, but it requires more stack operation. Another way to manage local variable is to get enough room for x and y in the first place, then we do not have to change ESP again. However, at label l1, we waste a few bytes. If x is not used once y is initialized, another solution is to use the same address for both. Globally, the point is that ESP may change and it is not reliable, but EBP do not move throughout the execution of the function and local variables are always allocated at the same place, relatively to EBP.

The idea is thus to refer to local variables relatively to EBP. Luckily, in our syntax for assembly blocks, the compiler provides a name for the offset between EBP and a local variable: it is the name of the local variable itself. This is illustrated in Listing 6.7:

x in assembly block is the offset (i.e. a compile-time constant integer value) between EBP and the address of C variable x. Thus, EBP + x is the address of the local variable x, then [EBP + x] is the local variable. It is written in the alternative form x[EBP] as a[b] is an alias for [a + b]. It is sometimes more user-friendly since it mimics the syntax for array access in C.

Sharing Symbols

We have seen how C and assembly can interact through data, using the same variables. There is another kind of interaction: through the control flow.

Or course, as usual, C functions can call C functions, and assembly statements may jump into assembly code. But there are also mixed possibilities: assembly can call a C function, and conversely, C code can call an assembly label just as if it was a C function.

Assembly code sees all C symbols. The converse is not true: C code does not see assembly labels, the only names defined by assembly. However, there is a trick illustrated on Listing 6.8. The assembly label f is not directly visible, but an external function with the appropriate type and the same name is declared (but not defined). This way, the C compiler allows C code to call f. Later, during linking, the reference to f is recognized extern void f(void); The way such pseudo-functions get their arguments and provide their return values will be explained in section 6.5 "Calling Convention" (page 87). To make C and assembly interact nicely, some characteristics of x86 must be applied to C. An important subject is the representation of values: they must be understood in the same way by C and assembly code. This property requires that endianness and representation of signed integers must be shared. Thus, C code uses little endianness and two's complement representation, like x86 processors.

Assembly-Induced Assumptions

These assumptions go beyond the C standard, but we should be careful about the implications. Two's complement representation allows nice wrap-around when integer overflow occurs in assembly. However, unsigned integer overflows are still undefined in C. If we want to perform an arithmetic operation with wrap-around on overflow, it can be done in assembly, where it is perfectly defined, just like in Listing 6.9. Otherwise, we need to assume the compiler defines this behavior, which does not imply (and once again, is not implied by) the two's complement representation.

Calling Convention

1 extern int g(int, int); A calling convention describes how functions get their parameters and how they should return their results. This is crucial to allow modular programming: functions defined in different files can call each other as long as they agree on the calling convention. In the same way, it allows code to be shared more globally, through libraries. The calling convention is thus a part of the application binary interface (ABI).

C developers should not really care about calling convention: they work at a higher level of abstraction, and the compiler is in charge of implementing the calling convention. But the calling convention is visible in assembly and to make assembly and C to work together, we have to manually implement it on the assembly side of calls.

There are many conventions. They depend, in particular, on features of the processor and the environment (e.g. the operating system). For instance, on 32-bit x86 processors, since there are few registers, parameters are usually given on the stack. On processors with more registers (like 64-bit x86 or other architectures), for performance reasons, we can pass parameters in registers. However, even for the given processor, Unix-like and Windows environments use different conventions. For instance, for 64-bit x86 processors, their respective conventions differ on the registers to use. There are usually no objective reasons to favor some registers over others, this is just a matter of history.

1 int g(int a, int b){ 2 //
The calling convention we are interested in is called cdecl (for "C declaration"). According to this convention, parameters are passed on the stack in right-to-left order, this means that parameters on the right (with respect to the C prototype) must be pushed first, and the leftmost one is the last pushed, and thus is the closest to the top. After the call, parameters are still on the stack and it is the responsibility of the caller to remove them. We say that this is a caller clean-up convention. It is very convenient for variadic functions: the callee does not have to be aware of the number of arguments, and their respective sizes, to infer the number of bytes to clean.

For small non-float types i.e. types that fit in 32 bits (such as integers or pointers), the returned value is stored in register EAX. We will not consider other cases where the returned type is a floating-point type or a type bigger than 32 bits, like aggregate types (array or structures).

Since EAX is used for the returned value, its content is obviously not preserved. But it is not the only one: EAX, ECX and EDX are caller-saved. This means that the called function (the callee) can use them freely and let them in any state at the end ‡ . On the other hand all other registers are callee-saved: the called function must let the registers in the same state as how they initially were (at the beginning of the call). So, the callee has two options: not to use them or to save them (usually on the stack) and restore them at the end.

To respect the calling convention, when an assembly function is called from C, we must check that the assembly callee preserves the value of all other registers. Conversely, when a C function is called from assembly, the value of caller-saved registers are unknown when the callee returns.

Examples of mixed calls are given on Listing 6.10 and Listing 6.11. An interesting fact shown by Listing 6.11 is that g cannot see ECX on the stack since it is too deep: g has 2 parameters, therefore everything deeper is hidden.

Both these examples show consistent returns: in Listing 6.10, the callee is an assembly function and it returns with the assembly instruction RET while in Listing 6.11, the callee is a C function that returns using a C return statement. It is absolutely required to do so: a C callee cannot return through an assembly RET instruction, and an assembly callee cannot return using a C return statement. Indeed, C functions have preludes and postludes that handle call frame management. If we call a C function, its prelude is run, some space on the stack is allocated, and ESP and EBP are updated accordingly. The postlude cleans up what the prelude did. But if we simply run a RET instruction after the prelude but without executing the postlude, it will not behave as wanted since at this point, the top of the stack is not the return address. Conversely, if we run the postlude in a context where the prelude has not been run, it will free some space that has not been allocated, and it will probably destroy the call frame of the caller. Even if C functions must end by a C return statement, there might be some assembly blocks in it, as long as they preserve the stack. ‡ For security reasons, for instance in cryptography libraries, the callee might be interested in resetting them anyway. It would be regrettable to leak private data through these registers. The same kind of leak may happen through the stack: if it is not reset, the caller may inspect the state of callee's local variables. Unfortunately, compilers rarely provides an easy-to-use and portable solutions.

Chapter 7

Pathological and Didactic Examples

T he semantics we will define in the next chapter is very general and is quite tricky.

Because the formalization may seem austere and not very explanatory, we will begin by giving examples that illustrate strange cases, justify why the semantics is that complicated, and why we do not exclude apparently undesirable behaviors.

One must keep in mind that these examples are not made up, they are all distilled versions of problems that appear in our study case.

Assembly Returning from Another Function

In this section, we will explain the Listing 7.1. For the sake of simplicity, all functions have type void(void).

We assume that the entry point of this snippet is f. C function f calls the pseudofunction * g. Then, in g, there is a jump to the label h that is in an assembly block in another C function. The ret instruction is contained in this function, and returns from the call.

The interesting point is that, in contrast with pure C, function enters and exits are not well-parenthesized: the last called function is not necessarily the first returning one. Here, we escape a function without a return. We can see this trick as a horizontal transfer of control.

Let us explain what happens in this snippet. When g is called, the caller (f) saves appropriate caller-saved registers and may initialize the call frame for the callee. Since g is an assembly label, nothing hidden is run, especially, no prelude. At this point, the stack is in the same state as how it was let by f at g() (line 18). The stack contains what f has set and the return address on top of it. The instruction jmp h just changes the current program point, thus the stack is preserved. The ret instruction pops the return address on the top of the stack and jumps to it, without running any postlude.

CHAPTER 7. PATHOLOGICAL AND DIDACTIC EXAMPLES

This return address leads to after the call of g and then, the return part of the calling convention is executed: registers are restored, stack is cleaned up and the remaining of the function runs normally.

Irregular Usages of Assembly RET

In the example of Listing 7.2, we call an assembly label, g, from a C function f, then we call the assembly label h from assembly. At this point, the top of the stack is the address after call h, under which, there is the return address of g(). Then, from h, we hack to return directly to f. The trick is to pop the top of the stack to expose the return address and perform the ret from there, skipping return to g. The surprising point that undermines the larger part of attempts to define the semantics, is that there is not the same number of CALL and RET. It is not the same as the previous case where the control is horizontally transferred since there still were, at least, the same number of CALL and RET instructions.

Another way to strangely use RET is shown on Listing 7.3. Here, the instructions push h and ret act like jmp h. Then, it is similar to the example shown in section 7.1 "Assembly Returning from Another Function" (page 91). The point is that the first RET executed does not return from a previous call. The control flow cannot be inferred only from the kind of instructions.

Dynamic Jump

With the previous examples, we have shown that the control flow is not as straightforward as one may think. But, at least, it was static: there was only one execution path possible. Even with conditional branching, the control flow graph can be inferred statically, like in C. Even function pointers have a very limited impact on the CFG, since it can be desugared using the transformation explained in subsection 4.4.2 "Desugaring C" (page 59): it adds relatively few edges, at most one from each call by pointer to each function with correct type. But assembly allows much more complicated control flows: destination of calls can be computed dynamically directly as an address, and not just as a selection of a label. Since each dynamic jump could lead anywhere, a directed acyclic graph (DAG) built without semantic analysis should have an edge from each dynamic jump to every other program point. A CFG is far less useful in such case.

The snippet of Listing 7.4 shows such a tricky example of dynamic jump. It is worth some explanations. Once again, the entry point is f, which has a parameter int n; this parameter will be forwarded indirectly to h. The C function g is not useful per se, but it hosts a big assembly block that contains only a repetition of call h. The first and the second are marked respectively with labels zero and one. To make this labels visible for C code, they are declared as external functions, but they are never called; they are only used in address computations. Under the assumption that all occurrences of call h are assembled in the same way (and thus have the same length), int d is initialized with the address of the n th occurrence.

When the n th occurrence of call h is called, the address of the next instruction is pushed on the stack (so the address of the (n + 1) th occurrence of call h) and control is given to h. In h, we copy in int d the return address and, by a similar computation as the one in f, we deduce which instruction was executed (in int n). Both local variables are static, so they are globally allocated, and they do not require the preamble to be run before exiting the function.

We can do computations involving n, they are skipped here, then we pop the stack to get the return address to f, and we return. If the stack is not popped before returning, the control will go the next call h and so on.

One should not forget this is a simplified version. There is a good (though incredibly complicated) reason not to directly pass n as a parameter of h in the real code. And, handling the real code is almost equivalent to handle this polished version.

This shows how complicated can be address computations and that we need to handle it with full generality. This also demonstrate a major difference with C and why assembly cannot be easily compiled to C and then handled for free in any tool that works on C. The C language does not have anything similar. Very close features are functions pointers and label as values (which is a GNU extension, so non-standard). But these features are not so flexible: because of the very nature of C, we cannot jump to a destination that have no name since we cannot predict how C statements will be compiled, and thus, the size they will take.

Sub-registers

This example shows a simple and very common hack that adapts very poorly in C. The trick is to get a part (in a bitwise meaning) of a value by writing a register and reading a sub-register. For instance, in Listing 7.5, short m is assigned with the 16 lower bits of int n. There is no complication here. The difficulty comes when one may try to adapt it brutally in C. A naive way is shown in Listing 7.6. It models the state of the register as a C structure, in which each register is a union, allowing to select the whole register or just sub-registers. But C does not behave this way. Writing regs.eax.EAX and then reading regs.eax.ax.AX is illegal: except if types of initial fields are compatible † , it is not allowed to read a field of a union type that have not been the last written. Indeed, there is no guarantee that the fields are located as we may think, for instance, AX may be located in the middle of EAX. Making the hypotheses required to make this model works may lead to accept some illegal behavior in pure C.

1 void f(int n) {
It may seem like a mere technical difficulty, but it is another argument that dissuades us to try to reduce assembly into C.

Peroration

We have seen various difficulties that undermine efforts to understand assembly as a C-like language: we cannot compile assembly into C (even enriched with registers), and we cannot assume assembly has a nice C-like control-flow structure. In particular, we cannot understand CALL and RET instructions as the C concepts of calls and returns, and we cannot assume calls and returns are well-parenthesized (or even in equal numbers).

Of course, this is not an exhaustive list of problems we found in our study case, but it is a sufficient subset to justify the choices of the following chapter. More importantly, we can handle them using relatively few (and reasonable) assumptions on the compiler and the environment. Without these assumptions, we could do almost nothing.

Our study case contains other issues that are not illustrated by these examples, and not quite handled by the following semantics. It can be easily adapted to handle them, but only using very strong assumptions. Since we want the semantics to stay as portable as possible, we will not add these assumptions from the beginning. Moreover, such code is not in the analyzed fragment of the study case, thus we may ignore these problems for now. Such problems, with solutions, will be explained in section 8.4 "Summary and Extension" (page 117).

Chapter 8

Semantics of Mixed C and Assembly

T his chapter is dedicated to a formalization of the semantics of mixed C and as- sembly. In previous chapters, we introduced all the concepts useful here, especially, the semantics of a C-like language, the semantics of assembly and what to expect from the semantics of the mix. The time has come to formalize this last semantics. This formalization is not so meant to be an explanatory approach of mix C and assembly, but to be the theoretical background for the following. As we want to analyze such code, we will design abstractions and formally establish their soundness. Yet, soundness is always relative to the concrete semantics of the language. This is what we will define here. Abstractions will be explained in Part III "Abstraction of Mixed C and x86 Assembly" (page 173).

The Formal Language

In chapter 4 "A Minimal C-like Language" (page 43), we defined Asclepius to be a model of C. We will now enrich it with assembly blocks. We call this enriched version Panacea * , for Plump Asclepius as a Neat Approximation of C Enriched with Assembly. Moreover, in ancient Greek mythology, Panacea (Πανάκεια) is the daughter of Asclepius and a goddess of universal remedy. With that name, we express the hope that Panacea is the final formalization solving all the problems, after a lot of trials and errors.

Definition 8.1 -Statements of Panacea

We extend the non-terminal symbol c_stat of the syntax of Asclepius (see definition 4.3 "Statements" (page 44)) as follows: We need to extend the labels of Asclepius to Panacea. For this purpose, we first need to define components of Panacea programs.

Definition 8.3 -Components Given π ∈ Π, we extend Cmp π : Stat → P (Stat × Π) of definition 4.5 "Compo- nents" (page 46) by Cmp π (asm {S 1 . . . S n }) := {(asm {S 1 . . . S n }, π)} ∪ n i=1 Cmp π.i (S i ) Cmp π ( asm_stat ) := {( asm_stat , π)} Cmp π ( asm_label ) := ∅ Likewise, we extend Cmp P : F [P ] → P (Stat × Π) f → Cmp (f,ε) (Body P (f ))
and

Cmp : P → P (Stat × Π) P → f ∈F[P ] Cmp P (f )
It is interesting to remark that labels are not components since they disappear at compile-time. Indeed, we do not want to put labels around assembly labels: assembly We can remark that the last part of paths of assembly statements separated by a label are not consecutive integers. For instance, in Listing 8.1 if the assembly block is located at path π ∈ Π, the instruction mov EAX, 1 is located at path π.1, and mov EAX, 2 is located at path π.3, but path π.2 does not exist since labels are omitted from components. This does not matter for the following since we never use paths explicitly, and it makes the formalization much simpler.

In addition to Asclepius labels (elements of L C ), we need to label assembly statements. Since assembly blocks are only a sequence of statements, we just need labels at the beginning of the block, at the end and between each pair of consecutive statements. We can identify such labels as an ordered pair (l, i) where l ∈ L C is the label of the assembly block (as an Asclepius component) and i ∈ N is the number of the program point in this block (which implies we skip labels). Formally, we define We need to add a labeling axiom to handle the case of assembly blocks.

Axiom 8.1 -Labeling of programs

We are given a program P ∈ P. We have the hypotheses of labeling of Asclepius programs (see axiom 4.1 "Labeling of programs" (page 47)).

CHAPTER 8. SEMANTICS OF MIXED C AND ASSEMBLY

Moreover, for C ∈ Cmp (P ), if C = asm {(a i ) i∈ 1,m }, π : in P (C) = {at P (C) , after P (C) , (at P (C) , 0)} ∪ m i=1 in P ((s i , ϕ(i))) {at P (C) , after P (C)} ⊆ L C ∀i ∈ 1, n , in P ((s i , π.ϕ(i))) = {at P ((s i , π.ϕ(i))) , after P ((s i , π.ϕ(i)))} ⊆ L asm ∀i ∈ 1, n , at P ((s i , π.ϕ(i))) = (at P (C) , i -1) ∈ L asm ∀i ∈ 1, n , after P ((s i , π.ϕ(i))) = (at P (C) , i) ∈ L asm
where (s i ) i∈ 1,n is the extracted sequence of a that keeps only the statements, that is, there exists n ∈ 1, m and ϕ :

1, n → 1, m increasing (i.e. ϕ is an extraction) such that ∀i ∈ 1, n , s i = a ϕ(i) ∀i ∈ 1, m , a i ∈ Stat asm ⇔ i ∈ Im (ϕ)
Even if the assembly block contains no instruction (it is empty or contains only assembly labels), there still is a label (in L asm ) inside. It is useful in the case where an assembly block only contains an assembly label: the Panacea label marks the program point that can be the destination of jumps.

Notation 8.1 -Labeled empty assembly blocks

We extend the notation 4.3 "Labeled statement" (page 47) for the case of empty assembly blocks. Given a program P ∈ P, and three labels

(k, l, m) ∈ L 3 if ∃π ∈ Π :    at P ((asm { }, π)) = l ∧ after P ((asm { }, π)) = m ∧ in P ((asm { }, π)) = {k, l, m} then we write l asm { k } m
This rule is much more simple and constructive than other labeling axioms. If we look at Listing 8.1 "Two instructions separated by a label" (page 101) again, and we call l the label before the assembly block, the program point before the first instruction will be (l, 0). The program point between both instructions is labeled (l, 1) (and is a synonymous for "a_label") and the last program point in the block is (l, 2). Labeling of assembly blocks is that simple because they have a flat structure.

Memory Model

We need a memory model that allows both pure C and pure assembly to be run together; it is necessarily a mix of memory models of both languages. There are two approaches: defining an almighty memory model to use in C and assembly, or using a memory model for C program points (where there are no registers) and another for assembly program points, and transiting between both when entering or leaving assembly blocks.

In subsection 4.3.1 "Memory Model" (page 48), we defined a C memory state as a 4-tuple containing a stack of local environments, a stack of return addresses, a global environment and a heap. In section 5.2 "Memory Model for Assembly" (page 65), we defined assembly memory states as an enriched heap, that models both the main memory and registers. In pure assembly, no distinction was made between registers, but in C, we need to distinguish general-purpose registers from others (see subsection 2.2.1 "Registers" (page 10)). Indeed, general-purpose registers are subject to be modified arbitrarily, while system registers (such as segment registers) are not modified by C code. The EFLAGS register is a special case: status flags may be modified but system and control flags are constant. In the first approach (an almighty memory model), we would keep general-purpose registers alive all the time, but assigning them arbitrary values at each step in C code. In the second approach (two specialized memory models), general-purpose registers would exist only in assembly blocks, but absent from the Cspecific memory model.

The correspondence between both approaches is quite trivial. Since C statements modify arbitrarily registers and are not accessible, we can forget it. Conversely, in a model without registers at C program points, we can consider they have arbitrary values. However, there is a non-obvious difference that favors a formalization based on a single almighty model: when an assembly block is immediately followed by another one, nothing is run between them, and value of registers are preserved, as illustrated on Listing 8.3. It is exactly as if there was only one block. To simplify this case, we choose to keep the registers at all times.

We thus keep the enriched heap (heap with registers). We also need an environment for global variables (the address of global variables). return labels. We cannot use directly the same kind of stack since an assembly can be called from C, just like a C function, but at the destination point, there is no local environment. On the other hand, the stack is explicit. The solution is to have a stack in memory that is divided into call frames. An assembly call frame is an explicit array, while a C call frame is a black box of unknown content. The only thing we know about C call frames is that they contain the local variables, but with unknown padding, order. . . . The call frame contains also other data, such as the return address, saved registers, etc. These bytes must stay unchanged not to risk corrupting the call frame, unlike local variables that can be safely modified, either modified by name or pointer.

First, we need to define the set of environments that allow variables declared with appropriate keyword to be stored in registers, that is

E := V (A A R )
A C call frame is a 4tuple (v, a, s, f ) where v ∈ E is the local environment, a ∈ L is the return address given as a label, s ∈ N is the size of the call frame and f ∈ F is the function whose prelude built this call frame. The content of an assembly call frame is explicitly defined in memory, and we just need to remember its size. Since call frames are contiguous, and we know the base address of the stack, we can compute the base address of each call frame. Both kind of call frames still miss a component to correctly handle mixed calls. For calls from C to assembly, we need to check that assembly restore the registers as required by the calling convention and touched anything in C call frames, except local variables. For calls from assembly to C, we need to be able to retrieve the original value of registers, as ensured by the calling convention. To perform these tasks, we keep a copy of the heap. Overall, call frames are elements of the set

((E × L × N × F) N) × H asm
Thus, the set of memory state is

M := E C × H asm × (((E × L × N × F) N) × H asm ) + × A × N × A × N × A L The first component (E C ) is the global environment, the second (H asm ) is the enriched heap. The third component ((((E × L × N × F) N) × H asm )
) is the sequence of call stacks with saved registers (in the H asm component). The fourth and fifth components (A×N) are respectively the base address of the stack and the size (counted downward) of the stack. The sixth and seventh components (A × N) are respectively the base address and the size of the code (which is a write-only zone). The eighth component (A L ) is a map that gives the memory address of each program point. These addresses must be contained in the memory block dedicated to the code and be consistent with instruction encoding, i.e. the distance between the addresses of consecutive assembly program points must be the size of a possible encoding for the instruction between these two program points. This question is detailed further in chapter 9 "Amical" (page 121).

We can remark that every C call frame has a return address, including the first one (the call frame of main function). This is to simplify this complex enough definition. We shall simply use an arbitrary value since it will never be used. We can also remark that the first call frame is always a C call frame.

Semantics

We will not present the semantics of Panacea with the same level of formalization as Asclepius. Indeed, it was already tedious and not so clear for Asclepius; for Panacea, it would be pantagruelian. The semantics is given in a more operational way. We always start by giving the intuition of what we shall do in the first paragraphs, before formalizing it. The reader can jump directly to the next title to skip formal details without missing any explanation.

Formally, we define the semantics of Panacea as a transition system whose states are ordered pairs of a label (in L) and a memory state (in M), that is, this is a transition system whose set of states is L × M.

We can distinguish cases according to the kind of statement we consider: some of them are easy.

Where it is Legal to Write

Global variables are always accessible and writable. But, in the case of local variables, this is not so straightforward. Local variables are located in the stack, but the stack contains many things: it is split in call frames. Assembly call frames are very explicit: we know what they contain. On the other hand, C call frames are guaranteed to contain local variables, but not only. Modifying local variables of deeper call frames is legal; these variables may be accessed through pointer. But writing in a C call frame, anywhere else is considered an immediate error (ω a : invalid address).

Lastly, it is illegal to write in the region of memory that contains code. Of course, the code, the stack and global variables cannot alias each other in any way.

We can formalize whether a region is writable. We start by a utility function that gives the size of a call frame.

cs_size : (E × L × N × F) N → N (_, _, n, _) ∈ E × L × N × F → n n ∈ N → n
Then we can easily check where a region of size s, starting at address a is writable in the memory state m.

is_writable : M × A × N * → {ff , tt} (m, a, s) → let ((c i , _)) i∈ 1,n := π 3 (m) in ∃v ∈ V : a, a + s -1 ⊆ π 1 (m) (v), π 1 (m) (v) + |v| -1 ∨ ∃i ∈ 1, n : c i ∈ N ∧ 0 π 4 (m) -a - i-1 j=1 cs_size(c j ) < c i ∨ ∃i ∈ 1, n : c i ∈ E × L × N ∧ let (e, _, _) := c i in ∃v ∈ V : a, a + s -1 ⊆ e(v), e(v) + |v| -1
It simply checks whether the given region is in a global or local variable, or in an assembly call frame. This check must be performed when getting the address of an lvalue (that is computing ⦉l⦊), when we intend to write it. If the result of is_writable(m, a, s) is ff , the transition gives only ω a .

Pure C and Pure Assembly Statements

We start by examining the case of pure C and assembly statement. The idea is to transform pure C or assembly transitions to Panacea transitions.

Pure Assembly Statements

In the set of assembly statements, this family includes non-control statements or control statements whose destination is in the current assembly block. For these kinds of statements, the transitions are very similar as the one for pure assembly, adapted for the new memory model. We assume we are given a program

P ∈ P. Let (m, m ) ∈ M 2 be two memory states such that ∀i ∈ {1} ∪ 4, 8 , π i (m) = π i (m ). Let (l, l ) ∈ L asm 2 such that there exists an instruction s ∈ Stat asm such that ∃l ∈ L asm : l s l . Let a := π 8 (m) (l) and s := π 2 (m) (l ) -π 2 (m) (l). Then the transition (l, m) → (l , m ) is valid if and only if: -the transition (π 2 (m) , a, (0 → s)) → (π 2 (m) , a, (0 → s))
is valid in pure assembly for the program containing only the instruction s at address a, -lvalues pass the is_writable check, -topmost call frames of m and m are assembly call frame, -(ESP ) (m) + s = (ESP ) (m ) + s where s (resp. s ) is the size of the topmost call frame of m (resp. m ); this guarantee the size of the call frame is updated correctly when a stack instruction occurs,

SEMANTICS

107

-EIP needs to be updated. Since it will be a common operation, we define a function to do that:

update_EIP : H asm × L × A L → H asm (h, l, pp) → h [EIP i → pp(l) [8i : 8i + 7]] 3 i=0 We shall have ∀i ∈ 0, 3 , update_EIP(π 2 (m) , l , π 8 (m))(EIP i ) = π 2 m (EIP i )
We shall be careful about RET statements. Indeed, we cannot syntactically check whether the return address is an assembly program point, since it is contained on the stack. If the return address is indeed an assembly address, the previous criterion is valid, otherwise, this is likely a return from a C-to-assembly mixed call, which will be explained later (see subsubsection 8.3.4.1 "Calls from C to Assembly" (page 110)). To know if the given return address ra is a regular return, we shall just test whether there is an assembly program point with this address, that is

∃l ∈ L asm : π 8 (m) (l) = ra
We may also go even further. If the address is between two program points in the same assembly block, the return address is at the middle of an assembly instruction. This means it cannot be a return from a mixed call and the RET is not valid. Formally, if

∃(l, l ) ∈ L asm 2 : ∃c = (s, π) ∈ Cmp (P ) : at P (c) = l ∧ after P (c) = l ∧ s ∈ Stat asm ∧ π 8 (m) (l) < ra < π 8 (m) (l )
then the return address is in the middle of an assembly instruction, and thus, we transit immediately to the error state ω a .

Pure C Statements

It is similar for Asclepius statements, the only non-trivial transitions are at the interface between languages: beginning and end of assembly blocks, and calls to assembly pseudofunctions. All other transitions are very close to the one we defined for Asclepius, but with a precaution: general-purpose registers may be used arbitrarily. Thus, to adapt Asclepius transitions to Panacea, we make them to modify arbitrarily all the generalpurpose registers. Indeed, not only registers are unknown in C, but they can change at each step. Let us formalize that, too. Let (m, m ) ∈ M 2 be two memory states such that ∀i

∈ {1} ∪ 4, 8 , π i (m) = π i (m ). Let (l, l ) ∈ L C
2 such that there exists an instruction s ∈ Stat C such that ∃l ∈ L C : l s l . Let h := π 2 (m) |A and h := π 2 (m ) |A be the nonenriched heaps. We assume that ∀r ∈ R, π 2 (m) (r) = π 2 (m ) (r) where R ⊆ A R is the set of non-general-purpose registers that are guaranteed to be untouched by C, that is R contains segment registers ((CS i ) i∈ 0,1 , (DS i ) i∈ 0,1 , . . .), system registers ((CR3 i ) i∈ 0,3 , . . .) and other special registers. Let ((c i , h i )) i∈ 1,n := π 3 (m) and ((c i , h i )) i∈ 1,p := π 3 (m ), be sequences of call frames without saved heaps. We assume that n-p ∈ -1, 1 and ∀i ∈ 0, min(n, p) , (c i , h i ) = (c i , h i ). Let ((v i , a i , _, _)) i∈ 0,α the sequence of C call frames in c and ((v i , a i , _, f i )) i∈ 0,β the sequence of C call frames in c . Then the transition (l, m) → (l , m ) is valid if and only if:

-the transition

l, (v i ) i∈ 0,α , (a i ) i∈ 1,α , π 1 (m) , h → l , v i i∈ 0,α , a i i∈ 1,α , π 1 (m) , h
is valid in pure Asclepius for P without assembly blocks (thus, with the same context of global and local variables), -lvalues pass the is_writable check, -label l belongs to function

f β ∈ F, - ∀i ∈ 0, 3 , update_EIP(π 2 (m) , l , π 8 (m))(EIP i ) = π 2 m (EIP i )
There is a special case to treat: returns. The problem is similar as the one for RET statements explained before. We can determine which C calls lead to a C function rather than an assembly label, since we do not have function pointers. But this is not the case for return statements, whose destination is known by the content of the call frame. The additional check to perform is that the return address in the current call frame must be a label in C code, and the function name in the current call frame must be the function containing the return statement. This is a protection to avoid returning from a function whose prelude has not been run. It is not possible to do so in pure Asclepius, but assembly gives the power to jump between functions. We will treat later the behavior if the return address is not in L C (see subsubsection 8.3.4.2 "Calls from Assembly to C" (page 114)). If the current function is not the one specified in the call frame, the program transits to ω i .

Entering and Leaving Blocks

Another family of transitions are very easy: entering and exiting assembly blocks. In fact, these transitions do not even involve C or assembly statements, they just update the current program point, without modification on the memory. This way, two consecutive assembly blocks behave just like one.

Entering Blocks

Formally, let us consider the component a asm{ b S 1 . . . S n c } d . We assume that the current program point is a, and the memory state is m. The memory state can be decomposed: g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp := m

We start by changing EIP to its new value:

h 1 := update_EIP(h, b, pp) 8.3. SEMANTICS 109
We craft a new assembly call frame, initially empty:

c p+1 := (0, h)
where h is an arbitrary heap chosen non-deterministically. Once we entered the block, the new state is (b, m )

where m := g, h 1 , (c i ) i∈ 0,p+1 , sb, ss, cb, cs, pp

Leaving Blocks

Exiting the block is slightly more subtle: it depends on whether the next transition is entering a block again. If not, in addition to changing the program point and updating EIP, we need to check if the assembly call frame is empty before removing it. We assume that the current state is (c, m). Let g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp := m and (size, _) := c p If size = 0, then the system transits to ω i . Otherwise, we update EIP:

h 1 := update_EIP(h, d, pp)
and remove the topmost call frame:

m := g, h 1 , (c i ) i∈ 0,p-1 ,

Bridging Blocks

If the next transition enters an assembly block (that is, there are two immediately consecutive assembly blocks), we only update the program point (and EIP), without removing the call frame. Likewise, to enter the next assembly block, we do not add a new call frame. There are two ways to do that. Either, we statically detect consecutive assembly blocks, and we merge them, or adapt the transition between them. Otherwise, we could keep the call frame when exiting the block, but we remove it lazily, if anything happens except entering a block. This is simpler to implement because it is very local: we do not have to check the next statements. But for the sake of the formalization, we prefer to do it statically. Thus, in the special case l } asm{ l where l ∈ L asm , in memory state m, we simply update EIP g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp

:= m h 1 := update_EIP(h, l , pp)
The new memory state is

m := g, h 1 , (c i ) i∈ 0,p , sb, ss, cb, cs, pp
and with the new program point, the new state is

(l , m )
It allows not only keeping the state of general-purpose registers, but also the stack suffix must not be empty when leaving a block to immediately enter another one.

Mixed Calls

Mixed calls allow C code to call an assembly function and assembly code to call a C function. This is a nice way to mix C and assembly; this is even the most common way to interface multiple languages, usually through a foreign function interface (FFI). This way of interfacing allows both languages to be strictly separated with explicit interaction (the call). As it is quite clean, we examine these cases now, and we will later see a more vicious interaction between C and assembly: inter-blocks jumps.

Calls from C to Assembly

Calling

When an assembly function is called from C, a size for the current call frame is nondeterministically chosen. Accordingly with the calling convention, the parameters are pushed on the stack in right-to-left order, followed by the return address. The return address is an arbitrary 4-byte integer that does not alias with any variable and is not located in an assembly block (i.e. between the address of the beginning of a block and the address of the end of the block). At this point, we make a copy of the current enriched heap in the caller's call frame. Here, general-purpose registers have arbitrary value but, for callee-saved registers, the same value must be restored at the return.

Formally, we have a call l v = f(e 1 , . . . , e n ) l in a memory state m where f is an assembly label. The precondition is (l, m). We call f ∈ L asm the Panacea label at assembly label f. Let us deconstruct m: g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp := m We start by adding a call frame of size 4(n + 1): 4 bytes for each parameter and for the return address. The base address of this call stack is

csb := sb - i=0 p cs_size(c i )
The call frame should be contained in the stack region of the memory, thus if csb -4(n + 1) sb -ss, then we transit to ω a . We assume we have adapted ⦉l⦊ and (e) to work with this memory model of type M. This is simply the given definition where we ignore irrelevant parts of the new memory state type.

h 1 := h [csb -4i + j -3 → (e i+1 ) (m) [8j : 8j + 7]] n-1,3 i=0,j=0
We need to add the return address. The return address ra is chosen non-deterministically. It should be consistent with pp: it must be greater than pp(l) and not greater than pp(l ). It is not necessarily pp(l ) since the caller might want to restore caller-saved registers before running the next instruction. Moreover, there are several constraints: it is located in the code region, that is in cb, cb + cs -1 , but which is not contained in an assembly block. That is,

∀c ∈ Cmp (P ) , ∃(λ, λ ) ∈ L asm 2 : λ s λ ∧ s ∈ Stat asm ⇒ ra ∈⊆ pp(λ), pp(λ )
The new heap is

h 2 := h 1 [csb -4n + j -3 → ra [8j : 8j + 7]] 3 j=0
The new call stack is an assembly one, thus, it is simply its size, with a dummy heap:

c p+1 := (4(n + 1), h)
where h is an arbitrary heap chosen non-deterministically. We save the current heap in the caller call frame, and for that we redefine c p :

c p := (π 1 (c p ) , h 2 )
Now, we need to update EIP register:

h 3 := update_EIP(h 2 , f , pp)
The new memory state is

m := g, h 3 , (c i ) i∈ 0,p+1 , sb, ss, cb, cs, pp
thus, the new state in the transition system is

(f , m )
Let us emphasize that this is not a single state: several values, such as the return address ra, are chosen non-deterministically. Thus, there is in fact a set of possible states.

Returning

Such a call can only be ended by a RET when the stack has the same height as at call time and the top of the stack is the return address, but the parameters may have been modified. Equivalently, we can also jump to the return address with a stack that is 4-byte shorter than the initial one.

A RET instruction with an assembly destination is performed as a jump (see subsubsection 8.3.2.1 "Pure Assembly Statements" (page 106) for intra-block jumps and subsection 8.3.5 "Other Control Statements" (page 116) otherwise) and, in any other cases, it fails in ω a .

At the end of the call, we check whether callee-saved registers have indeed the same value as initially, EAX register is used as returned value and the execution of the C function continues. If an assumption of the calling convention is not matched, the system makes a transition to ω i (illegal instruction).

Let us formalize this return. We run the statement l RET l , in the memory state m. Thanks to the criterion explained in subsubsection 8.3.2.2 "Pure C Statements" (page 107), we can tell if this is a return from a mixed call. If the return address is not a C program point, this is a classical assembly instruction. Let us deconstruct m: g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp := m

We also deconstruct the topmost call frame, this is necessarily an assembly call frame:

(size, _) := c p and we need the stack saved in the caller's call frame, which is always a C call frame:

(_, hs) := c p-1
We check that the stack has the correct height:

∀i ∈ 0, 3 , hs(ESP i ) = h(ESP i )
If this condition is not matched, the system transits to the immediate error ω i . If it is correct, we know the top of the stack contains 4 bytes for each parameter and 4 bytes for the return address. The state of parameters is irrelevant, but the return address must be unaltered: we shall check if the top of the stack is the same as in hs. The base of the call stack is

csb := sb - i=0 p -1 cs_size(c i )
We check whether

∀i ∈ 0, 3 , h(csb -4(size -1) -i) = hs(csb -4(size -1) -i)
If this condition is false, the return address has been altered, and the system transits to ω a . Now, we shall check that callee-saved registers are unchanged or have been correctly restored. We check whether

∀r ∈ {EBX, EDX, ESI, EDI, EBP}, ∀i ∈ 0, 3 , h(r i ) = hs(r i )
If this condition is not matched, the system transits to ω i . Now, we can perform the actual return. Let us determine the return program point. The 4 bytes at the top of the stack is the return address, that is

ra := 3 i=0 2 8i h(csb -4(size -1) -i)
Thanks to pp map, we find the appropriate program point, that is the program point rpp such that pp(rpp) = ra. We look for the call instruction just before this label, that is the call such that

_ v = f(e 1 , . . . , e n ) l
Such a call necessarily exists since a C return address is stored in the saved heap only when executing a call. We copy the returned value from EAX to v. The actual address of v is found in the global environment or the caller's call stack, we can use ⦉v⦊ for that purpose. We need a memory state where caller's local are visible. We build this state

m 1 := (g, h, (c i ) i∈ 0,p-1 , sb, ss, cb, cs, pp)
where the assembly call frame is removed. We can now update the return variable:

h 1 := h ⦉l⦊ (m) + i → h(EAX i+1 ) 3 i=0
The last thing to do is to change arbitrarily some registers. In particular, we change general-purpose registers and EIP. Let R := {EAX, ECX, EDX, EBX, EDI, ESI, EBP, ESP} an non-deterministically chosen map x : R × 0, 3 → I, and

h 2 := h 1 [r i → x(r, i)] r∈R,i∈ 0,3
We update EIP:

h 3 := update_EIP(h 2 , rpp, pp)
The resulting memory state is

m := g, h 3 , (c i ) i∈ 0,p-1 , sb, ss, cb, cs, pp
and the new state in the transition system is

(rpp, m )
Since this way of calling function does not go through a C function prelude, no local variables can be declared as long as the stack ends with an assembly call frame. Calling a C function from assembly is a solution to get back into a nice setup with local variables. In the same idea, since we do not want to run a postlude: an Asclepius return in a context where the current call frame is an assembly call frame is an immediate error (ω i ).

Calls from Assembly to C

Calling

The converse operation consists of an assembly CALL to a C function. The number of parameters of the callee is known from its prototype. If the assembly call frame is not deep enough (that is, it is smaller than the cumulative size of arguments), the system terminates with error ω i . Caller's CALL instruction is executed: it pushes the current return address on the stack and jump to the beginning of C function. At this point, before executing the callee, the heap is saved in caller's call frame.

Let us formalize that. We want to run the statement l call f l where f is a C function with n parameters called x 1 to x n , with respective types t 1 to t n Let us call l the first label in function f. We call m the current memory state. We ritually deconstruct m: g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp := m

The topmost call frame is an assembly one. We decompose it:

(size, _) := c p First, if size < n i=1
|t i |, the system transits to ω i , otherwise the execution continues. We need to build the new call frame. First, let us choose a size s ∈ N. It should be big enough to store local variables and the return address. The base address of the new call stack is

csb := sb - p i=0 cs_size(c i )
Let us initialize the new local environment:

e : {x i | i ∈ 1, n } → A x i → csb + 1 + i-1 j=1 |t j | Overall, the new call frame is c p+1 := ((e, l , s, f), h)
and we save the heap in the caller's call frame:

c p := (π 1 (c p ) , h)
Now, we simply need to make general-purpose registers unknown, just like before:

h 1 := h [r i → x(r, i)] r∈R,i∈ 0,3
where x : R × 0, 3 → I is a non-deterministically chosen arbitrary map. We update EIP:

h 2 := update_EIP(h 1 , l , pp)
The resulting memory state is

m := g, h 2 , (c i ) i∈ 0,p+1 , sb, ss, cb, cs, pp
and the new state in the transition system is (l , m )

Returning

Such a calls end with an Asclepius return statement. According to the calling convention, we copy back the original value of callee-saved registers from the saved heap to registers. The return value is copied into EAX and the stack is popped so that only the parameters are left. We can remark that the callee may have modified the parameters since they are local variables from its point of view.

Once again, we have to first check if this is a return from a mixed call, or a regular Asclepius call. This has been already explained: we just need to see if the return label is an assembly program point. Otherwise, see subsubsection 8.3.2.2 "Pure C Statements" (page 107).

We want to run l return e; l in the memory state m. Let g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp := m and ((v, rl, size, f ), _) := c p First, let us check we will indeed run the correct postlude: the label l must be in the function f . If is not, the system transits to ω i . If everything is correct, we may copy the returned value to EAX, as commanded by the calling convention.

h 1 := h [EAX i → (e) (m) [8i : 8i + 7]] 3 i=0
The calling convention also orders the callee to restore some registers. We define this set of registers: R := {EBX, EBP, ESP, ESI, EDI}

We also need the enriched heap saved in the caller's call frame:

(_, hs) := c p-1
Thanks to this saved heap, we can restore registers:

h 2 := h 1 [r i → hs(r i )] r∈R,i∈ 0,3
We now need to update EIP:

h 3 := update_EIP(h 3 , rl, pp)
The new memory state is

m := g, h 3 , (c i ) i∈ 0,p-1 , sb, ss, cb, cs, pp
and the new state is (rl, m )

Other Control Statements

In assembly, there is no fundamental difference between JMP, RET and CALL. We can break down the two latter as stack operations and a JMP. The stack operation may succeed or fail the same way as real stack operations. Thus, we will just consider the case of JMP.

A JMP instruction simply updates the current instruction pointer. Thanks to the mapping between assembly program points and addresses, we can deduce whether the destination is indeed a legal program point or if it is outside assembly blocks or in the middle of an assembly instruction. If the destination is not legal, this is an immediate error (ω a ). However, even if the destination is legal, the jump may lead to other problems. There are two cases: whether the destination is in an assembly block in the same function or not.

If the destination is in the same function, local variables may still be accessible. In fact, it depends on how the compiler allocates local variables. If they are all allocated at the beginning of the function, they are all accessible at any point of the body, but if they are allocated when declared, we can access local variables if the sets of local variables alive at the source and at the destination are identical.

To formalize that, we need stronger assumptions on the compiler. Unlike many assumptions made until now, these would not be reasonably generalizable. In our application, C is limited to ANSI C (C89), where local variables should be declared only at the beginning of the function: in the outermost local scope, before the first statement. Because of these assembly blocks, the compiler is very little optimizing, and local variables are always allocated; even if two variables have disjoint lifetime, they will not share a memory cell. The nice consequence is that all intra-function jumps are valid. Formally, to tun l jmp k l in memory environment m, with g, h, (c i ) i∈ 0,p , sb, ss, cb, cs, pp := m we simply have to update EIP

h 1 := update_EIP(h, k, pp)
where k ∈ L asm is the label at assembly label k. The new memory state is m := (g, h 1 , (c i ) i∈ 0,p , sb, ss, cb, cs, pp)

SUMMARY AND EXTENSION

and the new state is (k, m )

If the destination is in another function, since the stack is not updated, local variables of the destination function are not allocated since we did not run the prelude. Local variables of the source function are not in the current scope and thus cannot be accessed by their name; this is not the concern of the semantics: we can check syntactically the scope of variables. Thus, in the destination function, we can only access global variables, and, if the control eventually returns to the source function, the stack must be in the same state to ensure that the execution continues correctly.

Once again, we need extra hypotheses for such tricky jumps. The safer way to handle them is to forbid it. Sadly, this is not always possible: system developers' hands are more stained that Lady Macbeth's (see [START_REF] Shakespeare | The Tragedy of Macbeth. Act V, scene 1[END_REF]); they fear no hack to achieve their goals. In our case, we restrict the problem to functions that have no local variables † . Thanks to this restriction, we may handle inter-function jumps exactly like intra-function jumps. It is worth noticing that we allow static local variable, as shown in Listing 7.4 "A complicated dynamic jump" (page 96) since they are statically allocated, and do no depend on a prelude/postlude pair.

If we need to support functions with local variables, we must instrument the semantics to know whether local variables have been allocated: if they have not been allocated while an expression tries to access them, the system must transit to an error state.

Summary and Extension

We saw how to handle mixed code. We have chosen a single memory model compatible with both C and assembly code. We have seen that transitions can be categorized in a few classes:

-Intra-language transitions can be easily adapted to this mixed code.

-Entering an assembly block requires only to update the current program point and add an empty assembly call frame. Leaving a block is more subtle: if the next transition is to enter another one, we only update the program point, otherwise, we check is the current assembly call frame is empty, and we remove it. -Mixed calls is a lot of work, but ultimately, the interface is clean. This is very dependent on the calling convention. -Assembly jumps between blocks require very precise assumptions on the compiler.

With our assumptions, these cases are quite easy. States of the transition system are pairs of a program point (as an element of L) and a memory state. The memory state contains the value of EIP register (instruction pointer). Since the memory state also keeps the mapping between program points and addresses, having both is redundant. In fact, we need the label since it is a simpler way to specify the current program point, and we modeled EIP for completeness. In fact, EIP is never used by its own and will be ignored from abstractions. When its value is needed, we will only use the label symbolically. This is similar to pointers: we do not care about the actual value of a pointer (and we often do not know it), but only to which variable it points. In the same way, the content of EIP is useless; we are only interested to know which program point it designates. We have mentioned in section 7.5 "Peroration" (page 98) that some issues are not handled by this semantics. For instance, we cannot reach a deeper assembly call frame since we do not know the size of C call frames, and thus, the number of bytes to skip. For instance, in Listing 8.4, at line 3, we have a stack made of 4 parts (from the topmost call frame):

-an empty assembly call frame for the assembly block in g; -the C call frame of g; -the assembly call frame of the assembly block in f (8-byte long and containing 42 and the return address of the call); -the C call frame of f. From the assembly call frame in g, we would like to access the assembly call frame in f. It could be done easily using a pointer: if we had saved the value of ESP after line 9, it would still point to the desired value. But in the study case, it is simply done by using an offset relative to current EBP: we need to skip g's call frame, and the return address pushed by call g instruction. To do so, we need to know the size of call frame of C functions, even if they have local variables: for the sake of simplicity g does not in our example, but could. This behavior is easy to handle in the semantics: rather than choosing a non-deterministic value for the size of C call-frames, they get a statically chosen size. Of course, it requires more hypotheses on the compiler to be able to ensure that the size of C call frames of each function is constant, and to indeed predict it.

We can go slightly further and try to handle the access to deep C call frames, as illustrated on Listing 8.5. To access a local variable in a deep call frame, we need to precisely know the layout of C call frames. This is a very strong assumption, and it might be tempting to simply make all the stack explicit, up to a few opaque segments (like where register are saved) of known size. But if we do so, we may still ensure that C statements sees variables independently, otherwise, it would wrongly allow some undefined behavior, like array overflow. Just like knowing the representation of signed integers does not allow signed modular arithmetic (see section 6.4 "Assembly-Induced Assumptions" (page 86)), knowing the layout does not allow overflow. For instance, in Listing 8.6, even if we know that n is located just after a, the loop is still faulty. Indeed, the compiler can assume array overflow does not happen, and generate optimized code accordingly. For instance, since it sees the loop iterates over a, and performs at least 4 steps, it may infer that it clears all (and only) the array a, and generate an SIMD ‡ instruction to write 16 bytes at once, and not modify n at all.

1 void g(void) { 2 asm { 3 mov EAX, X[EBP]
; What should be X to retrieve n ? This particular problem is a bit strange to handle: for signed arithmetic with two's ‡ single instruction, multiple data complement, it is easy to distinguish two kinds of addition (wrapping around or not); it is simply a different function to call. Here, it is more technical to know the layout but forbid array overflow. For pure C statements, the concrete structure of the stack must be totally forgotten, and we keep only the stack of environments. But if the heap has still a constant structure, we will accept faulty programs such as Listing 8.6. So, in addition, we shall scramble the heap to randomize the address of variables: we have to put blinders on C not to allow extra behaviors. We did not solve these problems directly in the semantics since it requires stronger assumptions that we do not want to make mandatory. Moreover, it is not on the analyzed fragment of the study case, and we have no satisfactory abstraction to handle such codes.

Chapter 9 Amical D etermining the size of instructions is a crucial matter to infer the destination of dynamic jumps. This is the job of a library called Amical (for AsM Intel in oCAmL) * , we wrote specially for Astrée as part of this work. Moreover, it can disassemble byte sequences (see subsubsection 2.4.2.1 "A Syntactic Digression" (page 30)), even partly symbolic. This is useful when the opcode is manually specified in the source code using assembler directives. Amical also has a standalone version that takes assembly instructions and returns the possible encodings. This mode is very limited since there is no C context (e.g. providing variables), but very useful to debug.

Amical solves another problem: determining the operand size. While it can be written explicitly † , it is usually implicit. In fact, in our study case, such annotations are explicitly written only when they are required. For instance, the instruction

1 mov [EAX], 1
is ambiguous because there is no way to know how many bytes are dereferenced by the memory operand; we could write 1, 2 or 4 bytes at once. An assembler would fail to process it, for instance GNU Assembler 2.30 will emit the error message "ambiguous operand size for mov". If we want to write 4 bytes, we can write one of these possibilities:

-

mov DWORD PTR [EAX], 1 -mov [EAX], DWORD PTR 1 -mov DWORD PTR [EAX],
DWORD PTR 1 It is worth mentioning that this problem exists only with Intel's syntax, since AT&T's syntax makes the operand size a required suffix of the mnemonic. In AT&T's syntax, the previous example would be written:

1 movl $1, (%eax)
in which the suffix l (that stands for "long") specifies that the instruction has 4-byte operands. Nevertheless, the problem of instruction size still applies.

There is a major semantic difference between this instruction * Amical is french for "friendly". Indeed, Amical is a nice tool that takes care of a painful problem for us. The latter writes 1 in a single byte, while the former writes 1 in lower byte and 0 in the 3 other bytes. Thus, determining the size of memory operands is crucial. Sizes of register operands is trivial, since each register name comes with a size: 4 bytes for registers prefixed by E, 1 for registers matching [A-D][HL], 2 for the others.

Clearly, the size of memory operands is important since it has a semantic impact. But, maybe surprisingly, operand size of immediate operands is also crucial, and no more trivial to infer. For instance, the instruction add BL, 256 does not add 256 in BL: BL is a 8-bit register, and the immediate operand is expected to be 8-bit long as well. 256 does not fit in a single byte (it is 1 0000 0000B), thus, only the result of the modulo is kept in the binary: 0. This is different from adding 256 in modular arithmetic. Indeed, adding 256 always causes an overflow (and sets EFLAGS register accordingly), which never happens when adding 0. This is a reason why the size of immediate operands has a semantic impact as well.

Moreover, the size of an immediate operand is not always the same as the size of the other operand. For instance, the instruction add DWORD PTR [EAX], 1 can be assembled in different ways: the immediate operand can be 1-byte (and signextended ‡ ) or 4-byte. Such special rules exist to propose smallest encoding for usual operations § , like adding small numbers. But the instruction add DWORD PTR [EAX], 256 does not raise this issue: since 256 does not fit in a single byte, this instruction will be assembled using a 4-byte immediate operand. Since the destination is indeed 32-bit wide, the value of the immediate operand is not affected. While the operand size has no effect on the semantics in the penultimate case, it changes the legal encodings, and thus the possible total lengths. This is an example of how the problem of the instruction size and the problem of operands sizes are linked.

We can remark that these problems appear only because we are working on source code. If we were working with the binary, assembling is already performed, and thus operand size would be explicit and instruction lengths would have been already chosen by the assembler. On the other hand, in binary, syntactic structures have been lost, which ‡ That is, extended to the 32-bit encoding with the same value when interpreted as an integer with two's complement representation.

§ Smaller encoding allows smaller executable file size, and quicker decoding, thus faster execution.

THE X86 INSTRUCTION ENCODING

123 are very useful to analyze C. The question of simplified compiled program ¶ vs. original source code is a leitmotiv in program analysis. As a rule of thumb, we often choose the original source code since it contains all syntactic information used by heuristics to improve precision. Indeed, to use heuristics on compiled programs, we often need some steps of decompilation to try to retrieve original structures. Similar questions will be discussed later.

The x86 Instruction Encoding

Before explaining how assembling and disassembling work, we will explain the instruction format. The following does not include anything from the 64-bit generation as it is irrelevant here. There are up to 4 1-byte prefixes, at most one of each prefix class, and they are all optional. The 4 classes of prefixes are:

Prefixes

-Lock and repeat prefixes. They are mainly out of our scope but quickly discussed in section 16.5 "String Instructions" (page 204). -Segment override prefixes. They allow overriding the segment in which a memory operand is resolved, rather than using register-directed disambiguation. -Operand-size override prefix. It allows 16-bit parameters while working in 32-bit mode, and conversely. -Address-size override prefix. It allows 16-bit addressing style in 32-bit mode, and conversely. The opcode identifies the instruction. It is 1 or 2 bytes long. As the opcode defines the operation, it specifies what kind of parameters is expected and their size.

ModR/M and SIB are two 1-byte fields used to encode register and memory operands. These fields are optional since they are not needed when the instruction requires no such operand. It is important to clarify what can be encoded here. These bytes can store a register-only operand (denoted r) and an operand that can be either a register or a memory operand (denoted r/m). For instance, in the instruction mov EAX, EBX, there are two register operands, thus one of them is encoded as a register-only operand, and the other as a register-or-memory operand.

Displacement field can be 1, 2 or 4 bytes large (depending on ModR/M and SIB fields) and represent a literal offset in a memory operand.

Immediate field is also 1, 2 or 4 bytes large (depending on the opcode) and stores an immediate operand, that is a literal value.

Yet, there are irregularities to this general layout. Two of them are quite interesting in our scope. For some opcode, additional precision can be stored in the ModR/M byte. Specifically, this byte holds 3 bits to specify a register operand that may be used to precise the opcode, in which case, this instruction cannot have a register-only operand. The most important example is the case of opcodes 80H, 81H and 83H. Depending on the supplementary bits in the ModR/M byte, it can either be instructions ADD, OR, ADC (add with carry), SBB (subtract with borrow), AND, SUB, XOR and CMP. All these variants of these instructions have no register-only operand but a register-or-memory operand (r/m) and an immediate operand. The second irregularity is somehow the opposite: the opcode contains the argument. A classic example is the instruction DEC (decrement). There are two variants: one with a register/memory operand that has a regular encoding, and one with register-only operand. For the latter variant, the 5 upper bits of the opcode are 01001B and the 3 lower bits encode the register to decrement. Thus, 01001000H is DEC EAX, 01001001H is DEC ECX (sic), and so on. We can consider this family of instructions as 8 zeroary opcode to interpret this case as 8 regular encoding rules.

It is interesting to note that arguments are not ordered. For instance, the only difference between mov EAX, [EBX] and mov [EBX], EAX is the opcode, but the parameters will be encoded in the same way.

Assembling rules are provided as on Table 9.1. The first column describes the binary encoding: the first number is the opcode in hexadecimal and the following specifies the fields used in the encoding. For instance, ib, iw and id are respectively immediate arguments of size 8, 16 and 32. The specifier /r means that there is a ModR/M byte (and optionally a SIB byte and a displacement) to encode a register-only operand and a register-or-memory operand.

The second column is about the syntax of the assembly instruction. For instance, the first line of Table 9.1 allows AL as left-hand side operand and any 8-bit immediate operand on the right-hand side. The description column is for documentation purpose.

The correspondence between the argument and the encoding is not explicitly stated, but there is no ambiguity since each kind of argument may only have one of each kind. For instance, for the instruction add [EAX], EBX, the only possible is "ADD r/m32, r32". Since the ModR/M byte can encode exactly one register operand and one r/m operand, there is no ambiguity.

The assembling rule is subject to ambiguity. For instance, add EAX, EBX can be assembled using the rule "ADD r/m32, r32" or "ADD r32, r/m32". In the first case we result with encoding 01 D8 while, with the latter rule, we get 03 C3. Here, both encoding have the same size (2 B), so, in our case, it makes no difference since we are only interested in the size of instructions. But other cases may have several encoding of various sizes. We already gave the example of add EAX, 3 in section 5.5 "A More Symbolic Semantics" (page 77).

Assembling has another purpose: deciding the size of memory operands. The size of register operands is unambiguous, since it is specified by their name, for instance EAX is 9. 2 always a 4-byte operand while AX is a 2-byte one. But the size of memory operands may be unclear without context. One may specify it explicitly, it is even required in some cases. For example, the instruction add [EAX], 1 may be assembled using five rules (all rules of the form "ADD r/m?, imm?"). But these rules does not have the same semantics since they do not dereference the same size in memory, thus we should specify the size of the memory operand so as to decide which rule to choose. We can, for instance, write add DWORD PTR [EAX], 1 to specify that memory operands are 32-bit long and thus legal rules have the form "ADD r/m32, imm?". There are still two legal rules, but we can decide which to choose according to the immediate operand, and if both rules are still legal, they have the same semantics. But even when it is mandatory to specify the size, it may be not so trivial, for instance, we can write add [EAX], DWORD PTR 1, where we specify the size of the immediate operand which is enough to decide the size of the memory operand. Even when specifying the size is not required (when there is no ambiguity), it is not always obvious to decide the size since it depends on the mnemonics and the other operand (if applicable).

The following sections will explain how we exploit these rules to deduce the size of instructions and to disassemble binary sequences.

Assembling

Amical is in charge of deducing the size of instructions and memory operands. But it goes even further: it assembles completely so that it produces the final machine code. Indeed, once we have done everything to deduce the size of an instruction, computing the corresponding machine code is quite easy. Moreover, it helps to debug since we can see the binary sequences behind each size. In particular, we can use mainstream tools, to disassemble the sequence obtained with Amical, and check if it matches the input instruction. During Amical development, we assembled binary sequences using GNU Assembler 2.30 (using directive to manually define put bytes), and disassemble the resulting object file using GNU objdump 2.30.

Amical works in 4 mains step:

1. It selects all the assembling rules that match the mnemonic and the operands.

2. Among the rules selected previously, it excludes some of them according to additional assumptions. Here, it already knows the operand sizes according to the remaining rules. 3. It unifies the operands with the fields of the binary encoding. At this point, we can compute the size. 4. It puts the fields in the appropriate order to get the binary encoding. To select the assembling rules, we first select the rules with the appropriate mnemonic. This is really easy when the rules are stored in an associative container whose keys are the mnemonics and values are set of rules. This step is fast and restrains greatly the set of rules to examine.

For the remaining rules, we must look at each one independently. It is an acceptable cost since they are relatively few rules for a given mnemonic * * . For each rule, we check if each parameter of the actual instruction matches the corresponding operand pattern. For instance 1 can be imm8, imm16 or imm32, and EAX can be r32 or r/m32.

At this point, we have selected all the rules that match the instruction. It is important to get all rules because we do not know which one the assembler would choose. Indeed, while assemblers claim to always select the smallest encoding possible, this is not true † † . Sometimes this is just a bug, but sometimes it is because of alignment or to generate relocatable code (or any technical reason) ‡ ‡ . Yet there might be good reasons, it is hard to predict when the assembler will use a suboptimal encoding. Moreover, a longer encoding could be chosen because of a reason that may not always apply § § . So, to stay on the safe side, we must consider all possible encodings. However, sometimes, we have selected rules that are actually discarded by additional hypotheses. For instance, the instruction push [EAX], in 32-bit mode, pushes a 32-bit value on the stack, while in 16-bit mode (or 32-bit with operand size override), it pushes only a 16-bit value. The assembler assumes that it knows the current execution mode and that we want the default behavior, i.e. to push a 32-bit value. But Amical would have selected both rules "PUSH r/m16" and "PUSH r/m32". At this point, we discard the 16-bit version since assumptions of the assembler avoid an ambiguity here. More generally, Amical needs to know the current execution mode and it will only select rules that are available in this execution mode if there are any, discarding rules that require a operand-size override prefix. If there is no rule for the current execution mode, Amical keeps all 9.3. DISASSEMBLING 127 rules previously selected. This allows Amical to reject rules that would require an operand-size override that has not been explicitly required, while still accepting them if the operand size is constrained. For instance, the instruction push WORD PTR [EAX] explicitly specifies that the operand size is 16 bits. Thus, only the rule "PUSH r/m16" would match, and not "PUSH r/m32". Since they are no rule for the current execution mode, the other one is selected.

Here, we have selected all the rules we will work with. Since patterns of operands specify the size, we already know the possible sizes of all operands, but we still need to compute the size of the encoded instruction. For each rule, we determine which fields are allowed based on the specifiers of the "Opcode" column of Table 9.1, and infer how the arguments are arranged in these fields. Moreover, we must detect whether this rule needs an operand-size override or an address-size override prefix. This is mostly a matter of case analysis. We have here computed the content of all allowed fields in each rule. The sum of the sizes of all fields is the size of this encoding. But, to allow easier debugging, we put these fields in a row in the right order to give the correct instruction.

Amical is able to assemble instructions that contain labels and C variable names. Once again, it is only possible with additional hypotheses, e.g. the size of C variables must be provided. For labels, if they are in other blocks, we have no idea how far they are, and thus, whether the offset fits in a 8-bit, 16-bit or 32-bit integer, and we must consider that everything is possible. If the label is in the same block as its usage, we may compute possible distances as sums ¶ ¶ of possible sizes of all instructions in between. Thanks to the set of possible distances, we can discard some possibilities: when the label is at least 129 B away, the distance cannot fit in a 8-bit relative offset; or when it is at least 32 769 B away, the relative offset can only by 32-bit wide. Contexts containing the position of labels can be provided to Amical or computed automatically when it works on a sequence of assembly instructions and labels.

Disassembling

We already mentioned that we also need to disassemble binary sequences. This was briefly discussed in subsubsection 2.4.2.1 "A Syntactic Digression" (page 30) and illustrated on Listing 2.5 "Writing a far call using defines" (page 31). We will not linger on the reasons that lead the developer to write instructions in such a displeasing way; we will only explain how we solve that problem and find the correct instruction. Indeed, the internal representation of instructions in Amical can be as precise as the machine code (and thus more precise than assembly language), thus even instructions that must be written in binary in the source code can be expressed unambiguously in the internal format.

The problem of disassembling is much easier than assembling. First, we precisely know the size of the instruction. Moreover, finding the corresponding statement is very linear.

To disassemble, Amical takes a list of bytes (or symbolic bytes, such as labels). Then, they are consumed in very simple steps that follow the structure of instruction format as shown on Figure 9.1 "Instruction format" (page 123).

1. We read prefixes: they are 12 of them in 4 categories. Each category may appear up to once. 2. We read one byte of the opcode: if it is 0FH, the opcode is made of 2 bytes, and we read a second byte; otherwise the opcode is a single byte. 3. Based on the prefixes, the opcode and the current execution mode (which is a part of the configuration), we can find the corresponding assembling rule. Since different rules give different encodings, we are sure there is only one possible rule. 4. Using the specifier of the "Opcode" column, we know which fields are present: we fill them with the appropriate number of bytes. At this point it is interesting to remark that literal bytes, such as a label, must all be consumed at once and cannot be split. 5. From the value of each field and the form of the instruction, we can infer by which field is encoded each parameter, and thus we can rebuild the whole instruction. Leftover bytes belongs to the next instruction. Disassembling is really the easy way: there is only one involved rule, and we know the most precise representation, the binary sequence.

A more Concise Syntax for Assembling Rules

For each opcode, there can be many assembling rules, but they are often several of them that are very similar and follow quite regular patterns. Let us look again at the case of ADD instruction, the rules are given in Table 9.1 "Assembling rules for ADD" (page 125): there are 14 cases. But we can observe some regularity that is even documented in [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF]. First, 16-bit and 32-bit versions share the same opcode. Moreover, between the 8-bit and the 16/32-bit version, the difference is only the last bit of the opcode. When the two operands are writable (typically, a register-only operand and a register-or-memory operand), the direction is given by bit 1 (counting from 0), so between "ADD r32, r/m32" and "ADD r/m32, r32", only one bit in the opcode is flipped.

To be able to feed all rules to Amical without having to write too much manually (with the associated risk of mistakes, especially coming from copy-pastes), we developed a language to exploit patterns in assembly rules such that each line expand to several rules.

For instance, the 14 rules of ADD are reduced to 4 combined rules, as show on Table 9.2. The first, third and fourth lines expand into 3 rules each, and the second line expands to 5 rules.

The details are not relevant here, but we can give the overall idea. Since rules can differ from very few, we make these variations parameterizable. After the opcode, these parameters allow changing some bits, by default from the lower bits, but it can be specified otherwise. After a specifier or an operand pattern, it allows changing the size by using the "w" parameter (byte-size or full-size) and the "osa" parameter (operand There are 3 other parameters to allow rules to be compressed even more efficiently where possible. We will not detail everything and the subtleties of the syntax, but we can give a remarkable example: the case of conditional jumps. There are 16 kinds of conditional jumps and, as some of them have aliases, there are 30 mnemonics. Since each kind of condition is always encoded in the same way and use the same kind of suffix * * * , we can compress them efficiently. The result is shown in Table 9.3. The first line expand to 30 rules, while the second line, which has an additional parameter (16-or 32-bit), is expanded to 60 rules. To achieve such compression, we allow the mnemonic to be also parameterizable: the parameter tttn is understood as the encoding of the condition when it is part of the opcode, but in the mnemonic, it is interpreted as the condition name (and various aliases).

Of course, a rule without parameter will not be expended, and kept as is. This allows specifying rules explicitly, just like in [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF]. * * * Conditions are encoded in a 4-bit field called tttn in [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF].

Chapter 10

Introduction

T o guarantee the safety of a program we must check that all executions are correct.

As a consequence of Rice's theorem [START_REF] Rice | Classes of recursively enumerable sets and their decision problems[END_REF], this is impossible to do automatically. A solution to still check all executions without giving up the benefit of an automatic process is to check a bigger set of executions: by adding some not-actually-possible executions, we can make a bigger set on which the problem is decidable. The drawback is clear: if some impossible executions we added are faulty while all possible executions are error-free, we detect an error that cannot actually happen. Moreover, for every way to choose these additional impossible executions, there exists correct programs on which the analysis will unjustly detect errors. On the bright side, if the program is correct even with the extra traces, possible executions are necessarily also correct.

The way we add extra executions is not entirely artificial. Since we cannot automatically deduce the exact set of possible states of a program, we only work with a partial knowledge of possible states, thus allowing impossible states. This technique is called abstract interpretation. Before diving into mathematical details, let us illustrate abstract interpretation thanks to simple examples.

The most canonical example, given in [START_REF] Cousot | Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints[END_REF], is about the sign of the result of arithmetic operations given the sign of the operands. For instance, if an operand is negative and the other is positive, the result of the multiplication is negative. All the cases are summed up in Table 10.1. With a partial knowledge of the input, as we just know the sign but not the value, we can already deduce properties on the output. Of course, the result is also imprecise: we cannot get an exact result, but hopefully, we are only interested in the sign. the addition instead of the multiplication. We know that the sum of two positive (resp. negative) numbers is positive (resp. negative). But when a term is positive and the other is negative, we cannot say the sign of the result in general. In addition to the sign, we would need to know which has the bigger absolute value. The rule is given by Table 10.2. In this table ? means that we cannot say anything about the result: it can be positive, negative or null, i.e. the partial knowledge of the input is not precise enough to deduce anything. This special value (the complete absence of information) is usually denoted . We can remark that for the addition, if one of the term has an unknown sign, we cannot tell anything on the result. This is not an intrinsic property of , for instance, the product of ? and 0 is always 0. Nevertheless, getting this most imprecise value is often a bad omen, and it usually spreads quickly. When we lose all precision, we cannot deduce anything non-trivial (something that is not always true or always false).

× + 0 - + + 0 - 0 0 0 0 --0 +
+ + 0 -? + + + ? ? 0 + 0 -? -? --? ?

? ? ? ? The partial knowledge of a value is called an abstract value. Each abstract value represents a set of possible numbers, for instance + stands for all positive numbers, i.e. N * (or {1, 2, 3, 4, . . .}). The interpretation of an abstract value is the concretization. Some abstract values are more precise than others, for instance ? (that stands for all integers i.e. Z) is coarser than +; but some are not comparable, like + and -. We can present the allowed abstract values as on Figure 10.1: the higher an abstract value is, the less precise it is. At the very bottom, the special value ⊥ stands for the set of all integers that are simultaneously positive, negative and null: this represents the empty set, since no concrete value matches this constraint. Such a representation is called a Hasse diagram.

Another simple example is searching a word in a dictionary. The standard method is a binary search: we start somewhere in the dictionary (usually, hinted by the rank of the first letter in the alphabet, but this is not a requirement), and depending on the word we see, we know if the one we search is before or after. The principle behind is that we know that all the words that come before a given word are a subset of all the sequences of letters that are smaller (according to lexicographic order) than the given word. We do not need to know the exact set of words: we may not know all the words of the dictionary or, on the contrary, some words may be missing. But if the word we search is not in the huge set of all sequences that would come before, it can only be after (if it is in the dictionary). Here, concrete values are sets of words and abstract values are properties of the form "all words before (resp. after) the word w".

Abstract interpretation can be also found in disguise in many computer science methods. For example, let us consider the maximum satisfiability problem (MAX-SAT): given a formula in conjunctive normal form, what is the maximum number of clauses that can be satisfied by an assignment of Boolean variables, or alternatively, is it possible to satisfy at least K clauses (for a given K). This problem is well-known to be NP-complete, so we do not know polynomial algorithm to solve it, but we can strive to speed up resolution algorithms. Exhaustive search for a formula with V Boolean variables requires trying 2 V assignments, which is quickly prohibitive. Fortunately, this may be improved easily. We use by a DPLL-like algorithm [START_REF] Davis | A machine program for theorem-proving[END_REF]: it works by selecting an unassigned variable * , assigning it a truth value, simplifying the formula, proceeding recursively to find a solution, and eventually backtracking to try the other truth value for the selected variable. From a partial assignment, we can tell if a clause is satisfied (if at least a literal is tt), refuted (if all literals are ff ), or has still an unknown truth value (if all assigned literals are ff and there is at least one unassigned literal). Let us assume that we have already found a solution that satisfies m clauses, but we have not yet finished the search. Let us also assume that with the current partial assignment, i clauses are satisfied, j are refuted and l have still an unknown truth value. In the best case, at most i + l clauses can be satisfied by completing the current partial assignment. If i + l m, it is useless to continue the search with this partial assignment: it cannot give a better solution. Here, without knowing exactly which clauses will be satisfied by deeper search, we can tell whether the current state might lead to a better solution (otherwise, it definitely cannot), allowing us to reduce the search space. This is an example of branch and bound algorithm. The common idea in all these algorithms is to cut some branches of a search tree by recognizing early when an inner node cannot have leaves that beat the current best solution. This can be understood as abstract interpretation as inner nodes encode partial knowledge of a candidate solution † , and we only want to deduce bounds on the objective function.

In the context of program analysis, we want to approximate the semantics of the program: we run the program using imprecise states. At each step, from an approximate description of the precondition, we compute an approximation of the postcondition. It allows us to decide whether forbidden values (like a null variable just before dividing by this variable) are indeed avoided or if we cannot exclude them (though it does not mean they are really possible.).

The following of this part will explain the bases of abstract interpretation in general. First, we will explain the theory of abstract interpretation, from the order theory. We will focus on parts that are useful afterwards (in this part and the following ones), we will not give a comprehensive overview of abstract interpretation. Since the general theory does not provide an effective way to build abstract analyzers, we will then study the construction of complex abstract domains by product and reduction. Finally, we will explain the general architecture of Astrée, and some implementation details relevant for the following, either to explain some abstraction choices, or to give some context to subsequent comments on the implementation.

Chapter 11

Abstract Interpretation Theory T his chapter explains bases of abstract interpretation. The erudite reader that is aware of this matter may skip it safely.

Ordering and Lattices

This introductory chapter may be quite dry, but it requires little background. We start by elementary definitions and properties of order theory.

Definition 11.1 -Poset

A poset (for partially ordered set) is a tuple (E, ) where E is a set and an order relation over E, that is a relation which satisfies:

-reflexivity: ∀x ∈ E, x x -antisymmetry: ∀(x, y) ∈ E 2 , (x y ∧ y x) ⇒ x = y -transitivity: ∀(x, y, z) ∈ E 3 , (x y ∧ y z) ⇒ x z

In abstract interpretation, elements of posets are usually properties about the state of a program (or its execution trace). If an element x is smaller than an element y, then x is more precise than y, x carries more data. Indeed, it is always easy to say that anything can happen (the biggest element), while knowing the exact values of each variable is way more interesting.

Proposition 11.1 -Duality of posets

Let E = (E, ) be a poset. Let the relation defined by:

∀(x, y) ∈ E 2 , a b :⇔ b a
The tuple (E, ) is a poset, called the dual poset of E.

Proof.

-Reflexivity. Let x ∈ E. We have x x, thus x x.

CHAPTER 11. ABSTRACT INTERPRETATION THEORY

-Antisymmetry. Let (x, y) ∈ E 2 . We assume that x y ∧ y x. Thus, y x ∧ x y, then x = y. -Transitivity. Let (x, y, z) ∈ E 3 . We assume that x y ∧ y z. We have y x ∧ z y, thus z x and x z.

The notion of duality allows theorems to be easily transposed when reversing the order relation.

Notation 11.1

Given a poset E, we denote E the dual poset of E.

Proposition 11.2 For all poset E, E = E Proof. Let E = (E, ) a poset. The dual is E = (E, ) where
∀(x, y) ∈ E 2 , a b :⇔ b a
The dual of the dual is E = (E, ) where

∀(x, y) ∈ E 2 , a b :⇔ b a Thus ∀(x, y) ∈ E 2 , a b :⇔ a b Notation 11.2
Let (E, ) a poset, A ⊆ E, and x ∈ E. We denote

A ↓ := {x ∈ E | ∃y ∈ A : x y} A ↑ := {x ∈ E | ∃y ∈ A : y x} = A ↓ x ↓ := {x} ↓ x ↑ := {x} ↑
This notation will be useful in Part IV "A Reduced Product with Ghost Variables" (page 229).

Definition 11.2 -Total order

Let (E, ) be a poset. We say that is a total order if ∀(x, y) ∈ E 2 , x y ∨ y x 11.1. ORDERING AND LATTICES 139 Posets are rarely totally ordered in abstract interpretation, this is a much too strong structure. In a totally ordered set, there are only various levels of precision, we cannot express two unrelated states, with the incomparable precision. Total orders are useful only on subsets of posets. Such orders appear (indirectly) in the hypotheses of theorems about loop stabilization (fixpoint theorems). This is because the semantics of loops keeps previous iterations, thus we get an increasing sequence of states, that is, a sequence of states less and less precise, totally ordered. Let (E, ) be a poset, A ⊆ E and x ∈ E.

Definition 11.3 -Extremum Let (E, ) be a poset, A ⊆ E and x ∈ A. x is a minimum of A if ∀y ∈ E, x y x is a maximum of A if ∀y ∈ E,
x is an upper bound of A if ∀y ∈ A, y x

x is a least upper bound of A if

(∀y ∈ A, y x) ∧ (∀z ∈ E, (∀y ∈ A, y z) ⇒ x z)
Hence, the least upper bound is, as indicated by its name, an upper bound that is less than any other upper bound. We see in the next result that when it exists, it is unique hence it can be called the least upper bound. In abstract interpretation, when we know that either the property x or y is true, any upper bound of {x, y} is correct (since it is less precise than x and y), and the least upper bound is the best of these approximations.

Proposition 11.4 -Unicity of least upper bound

Let (E, ) be a poset, A ⊆ E and (x, y) ∈ E 2 . If x and y are least upper bounds of A, then x = y (i.e. the least upper bound is unique).

Proof. Since x is a upper bound of A and y a least upper bound, then y x. Symmetrically x y. Thus, by antisymmetry of , x = y.

Notation 11.4 -Least upper bound

When it exists, the least upper bound of A in the poset E is denoted E A.

We can omit the subscript, and chose a notation that matches the symbol for the order of the poset. For instance, if the order is denoted ⊆, the least upper bound can be noted ∪; if the order is denoted , the least upper bound may be written ∨.

Definition 11.5 -Greatest lower bound

Let (E, ) be a poset, A ⊆ E and x ∈ E.

x is a lower bound of A if ∀y ∈ A, x y

x is a greatest lower bound of A if

(∀y ∈ A, x y) ∧ (∀z ∈ E, (∀y ∈ A, z y) ⇒ z x)
We can remark that the greatest lower bound of A in a poset E is the least upper bound of A in E. The meaning in abstract interpretation is also dual: if a lower bound of {x, y} is true, then x and y are both true. In particular, the greatest lower bound is the coarsest value that allows this deduction.

ORDERING AND LATTICES

141 Proposition 11.5 -Unicity of greatest lower bound Let (E, ) be a poset, A ⊆ E and (x, y) ∈ E 2 . If x and y are greatest lower bounds of A, then x = y (i.e. the greatest lower bound is unique).

Proof. Similar to proof of proposition 11.4, by duality.

Notation 11.5 -Greatest lower bound

The greatest lower bound of A in poset E is denoted E A.

The comment about the notation of the least upper bound applies here also.

Definition 11.6 -Complete lattice

A complete lattice is a poset E = (E, ) such that ∀A ⊆ E, E A exists. Proposition 11.6 -Alternative definition Let E = (E, ) a complete lattice, thus ∀A ⊆ E, E A exists. Proof. Indeed, E A = E {y ∈ E | ∀x ∈ A, y x}.

Proposition 11.7 -Duality of complete lattices

Let E be a poset. If E is a complete lattice, so is E.

Proof. By duality, assuming that the greatest lower bound always exists implies that the least upper bound exists. This is an alternative definition of complete lattices.

Proposition 11.8

Let E = (E, ) be a complete lattice; E is not empty.

Proof. ∅ ⊆ E, thus E ∅ exists and belongs to E.

Notation 11.6 -Extrema in a complete lattice

In a complete lattice E = (E, ), we denote

⊥ := E ∅ = E E and := E E = E ∅
Thanks to these properties, we can prove many properties on complete lattices. They are very nice structures for abstract interpretation: the least upper bound is useful when a program point is accessible from multiple points, greatest lower bound allows refining a state given a known constraint.

Definition 11.7 -Lattice

A lattice is a poset E = (E, ) such that ∀(x, y) ∈ E 2 , E {x, y} and E {y, x} exist.

Proposition 11.9

Every complete lattice is a lattice.

Proof. Indeed, if the least upper bound exists for any subset, it exists a fortiori for any pair.

Lattices are weaker than complete lattices, some theorems do not hold on them, but some structures used in abstract interpretation may not have the structure of a complete lattice.

Definition 11.8 -Monotone maps

Let (E, ) be a poset and f : E → E. f is monotone if and only if

∀(x, y) ∈ E 2 , x y ⇒ f (x) f (y)
In semantics, monotone maps are a hypothesis of many theorems, and fortunately, they are very common. Indeed, it is very natural to think that if the precondition is less precise (greater), the postcondition will be less precise as well. It would be odd to gain precision on the result by losing some on the input. More inputs means more outputs.

We can note that this is true for the concrete execution, but not necessarily for an approximation of the execution. For instance, if approximate values are in a subset of concrete values, we can approximate a state by picking any bigger one that is an allowed approximate value. If the state is itself an allowed approximation, there is nothing to do. But if it is not, we may be very bad at finding an approximation, and not necessarily find the better one: with a slightly more precise input, we can obtain a worse output. Definition 11.9 -Chain Let (E, ) be a poset. Let A ⊆ E. We say that A is a chain if min A exists and (A, ) is totally ordered.

Definition 11.10 -Continuous maps

Let E = (E, ) be a poset and f : E → E. f is continuous if and only if

∀ chain A ⊆ E, E f (A) exists whenever E A exists and f ( E A) = E f (A)
: E → E. f is contractive if ∀x ∈ E, f (x) x f is extensive if ∀x ∈ E, x f (x)
Definition 11.12 -Fixpoint

Given (E, ) and a map f :

E → E. Let x ∈ E. -If f (x) = x, then x is a fixpoint of f . -If x f (x), then x is a pre-fixpoint of f . -If f (x) x, then x is a post-fixpoint of f .
Notation 11.7 -Fixpoint

Let (E, ) be a poset and f : E → E, we denote fp f the set of fixpoint of f , that is:

fp f := {x ∈ E | f (x) = x} Let x ∈ E.
We denote lfp x f the least fixpoint of f greater than x if it exists, that is:

lfp x f := min {y ∈ fp f | x y}
The least fixpoint of f is denoted lfp f , that is:

lfp f := lfp ⊥ f
Dually, we denote gfp x f the greatest fixpoint of f lower than x if it exsits, that is:

gfp x f := max {y ∈ fp f | y x}
The greatest fixpoint of f is denoted gfp f , that is:

gfp f := gfp f
Fixpoints are a central problem in analysis: they are useful in loops semantics. Nevertheless, we know that loop termination is undecidable, and equivalently, there is no automatic way to find a fixpoint in finite time. Thus, we shall focus on finding approximation of fixpoints.

Lemma 11.1

Let E = (E, ) a complete lattice and f : X → X a monotonic map. Then, lfp f and gfp f exist. Moreover,

lfp f = E {x ∈ E | f (x) x} gfp f = E {x ∈ E | x f (x)} Proof. Let P := {x ∈ E | f (x)
x} and m := E P . We will prove that m = lfp f .

∀x ∈ P, m x by definition of E ⇒ ∀x ∈ P, f (m) f (x) since f is monotonic ⇒ ∀x ∈ P, f (m) f (x) x since x ∈ P Since f (m) is lower than everything in P , by transitivity, f (m) m. f (m) m ⇒ f (f (m)) f (m) since f is monotonic ⇒ f (m) ∈ P by definition of P ⇒ m f (m)
since m is a lower bound of P By antisymmetry, we have m = f (m), i.e. m ∈ fp f . Moreover, we have fp f ⊆ P . Since m is the greatest lower bound of P and is a fixpoint, m = lfp f . Dually, we prove the existence of gfp f . Theorem 11.1 -Tarski's theorem Let E = (E, ) a complete lattice and f : X → X a monotonic map. fp f is a complete lattice.

Proof. Let A ⊆ fp f , we prove that lfp E A f exists.

Consider

F := {x ∈ E | E A x}. (F,
) is a complete lattice and f (F ) ⊆ F since f is monotonic. We consider g := f |F . Thanks to lemma 11.1, so g has a least fixpoint, and lfp g = lfp E A f . lfp E A f exists and is smaller than any fixpoint that is an upper bound of F .

Dually, we can construct gfp E A f . Thus (fp f , ) is a complete lattice. The lowest element is lfp f and the bigger is gfp f . Least upper bound application is X ∈ P (fp f ) → lfp E X f and greatest lower bound application is X ⊆ fp f → gfp E X f .

GALOIS CONNECTIONS FRAMEWORK

145

Galois Connections Framework

Abstraction can be formalized in many frameworks; among them we highlight two: concretization-only or Galois connections. Both formalisms will be explained as they have their own advantages and drawbacks. Eventually, we will work rather with the concretization framework, but we explain Galois connections as it is a very popular formalism and it is interesting to understand its limitations. This section is dedicated to abstractions through Galois connections.

Definition 11.13 -Galois connection

A Galois connection is a tuple ((C, ), (A, ), γ, α) where:

-(C, ) is a poset called concrete poset, -(A, ) is a poset called abstract poset, -γ : A → C, -α : C → A, such that ∀(a, c) ∈ A × C, α(c) a ⇐⇒ c γ(a)
Galois connections are defined using posets. Yet, many theorems need stronger structures, like lattices, CPOs (see definition 11.15 "CPO (complete partial order)" (page 150)) or complete lattices.

Notation 11.8 -Galois connection

A Galois connection ((C, ), (A, ), γ, α) is denoted

(C, ) γ ← ----→ α (A, )
A Galois connection is a tight correspondence between the abstract and the concrete world. It means that each concrete element c has a best abstraction α(c). In other words, α(c) is smaller than every other sound abstraction of c.

A Galois connection is a powerful tool, but it may not be the adequate framework. Indeed, many abstract domains do not define a Galois connection. While there are real-world such domains, it is easy to build one artificially: we just need to duplicate an arbitrary abstract element into a fresh (and not comparable) element with the same concretization. These two abstract elements are equally good abstractions of their concretization, thus there is no best abstraction anymore. -

α is monotone, -γ is monotone, -α • γ is contractive, -γ • α is extensive, are true if and only if (C, ) γ ← ----→ α (A, )
Proof. (⇒) We assume that α and γ are monotone, α • γ is contractive and

γ • α is extensive. Let c ∈ C and a ∈ A. We assume that α(c) a. α(c) a ⇒ γ(α(c)) γ(a) since γ is monotone ⇒ c γ(a)
since γ • α is extensive and transitive Dually, we can prove than c γ(a) ⇒ α(c) a.

(⇐) We assume that (C, )

γ ← ----→ α (A, ). Let c ∈ C. c = c ⇒ α(c) = α(c) Leibniz's law ⇒ α(c) α(c) Reflexivity of ⇒ c γ(α(c)) Definition of Galois connection Thus γ • α is extensive. Dually, we can prove that α • γ is contractive. Let (a, a ) ∈ A 2 .
We assume that a a .

α • γ(a) a Contractivity of α • γ ⇒ α • γ(a) a Transitivity of ⇒ γ(a) γ(a )
Definition of Galois connection Thus γ is monotone. Dually, we can prove that α est monotone as well.

GALOIS CONNECTIONS FRAMEWORK

147 Proposition 11.12 -Iteration of adjoints

Let (C, ) γ ← ----→ α (A, ). We have -γ • α • γ = γ -α • γ • α = α
Proof. We will only prove the first point. The second can be obtained by duality.

Let c ∈ C, a = α(c) and c = γ(a). By reflexivity of , we have c γ(a). By the definition of Galois connections, we have

α(γ(α(c))) = α(c ) a. Moreover, γ • α is extensive. Thus, c γ • α(c). And since α is monotone, α(c) α • γ • α(c).
By antisymmetry, γ

• α • γ = γ.
Definition 11.14 -Soundness Given a Galois connection (C, )

γ ← ----→ α (A, ), we say that a ∈ A is a sound approximation of c ∈ C if c γ(a) Let f : C → C and f : A → A. We say that f is a sound approximation of f if f • γ γ • f More generally, let n ∈ N. Let f : C n → C and f : A n → A. We say that f is a sound approximation of f if ∀(a 1 , . . . , a n ) ∈ A n , f (γ(a 1 ), . . . , γ(a n )) γ • f (a 1 , . . . , a n )
Here, we have tacitly lifted pointwise:

f • γ γ • f means that ∀a ∈ A, f • γ(a) γ • f (a)
We will do so in the following.

Proposition 11.13 -Alternative definition of soundness

Given a Galois connection (C, )

γ ← ----→ α (A, ), f : C → C and f : A → A. We have f • γ γ • f ⇐⇒ α • f • γ f Proof. (⇒) We assume that f • γ γ • f . Let a ∈ A. f • γ(a) γ • f (a) ⇒ α • f • γ(a) α • γ • f (a) since α is monotone ⇒ α • f • γ(a) f (a) since α • γ is contractive and transitive (⇐) We assume that α • f • γ f . Let a ∈ A. α • f • γ(a) f (a) ⇒ γ • α • f • γ(a) γ • f (a) since γ is monotone ⇒ f • γ(a) γ • f (a) since γ • α is extensive and transitive
This introduction may justly seem a bit arid and inelegant. The goal is to quickly come to the requirement of an abstraction to be able to state the interests and drawbacks of this framework.

To build an abstraction, we would like a way to abstract soundly loops that are defined by a least fixpoint. For this purpose, there are various fixpoint transfer or fixpoint approximation theorems.

To actually build an analyzer, we expect the abstract semantics to be inductively defined. In particular, if we denote S the semantics of a statement S, and S its abstract semantics, as we have S 0 ; S 1 = S 1 • S 0 , we need to have

S 0 ; S 1 = S 1 • S 0 .
The first theorem is very strong: under constraining assumptions, the least fixpoint of the abstract transfer function is the abstraction of the least fixpoint of the concrete transfer function. This is much stronger than stating it is a sound approximation.

Theorem 11.2 -Exact fixpoint transfer Let (C, ) γ ← ----→ α (A, ) be a Galois connection between complete lattices. More- over, let f : C → C, f : A → A, c ∈ C and a ∈ A such that -f is continuous -f is monotone -α • f = f • α -α(c) = a then α(lfp c f ) = lfp a f
We will not prove this theorem since we will not use it: its assumptions are not realistic. The requirement of f being monotone is generally not matched, especially when abstracting loops. Moreover, the requirement α • f = f • α is also utopian: asking f to preserve best abstractions is not realistic for expressive abstractions.

Even if we relax this last point, the resulting theorem is not so usable.

Theorem 11.3 -Approximate fixpoint transfer Let (C, ) γ ← ----→ α (A, ) be a Galois connection between complete lattices. More- over, let f : C → C, f : A → A, c ∈ C and a ∈ A such that -f is continuous -f is monotone 11.3. CONCRETIZATION FRAMEWORK 149 -α • f f • α -α(c) = a then α(lfp c f ) lfp a f Even the requirement α • f f • α is too strong in practice. Assuming we have α • S 0 S 0 • α and α • S 1 S 1 • α. α • S 1 • S 0 S 1 • α • S 0 by hypothesis S 1 • S 0 • α
by hypothesis and if S 1 is monotone Thus, for the abstraction to be compositional, we need all the elementary abstract transfer functions to be monotone, and not just the compound transfer functions of loop bodies.

To summarize, the Galois connection framework requires to have a best abstraction map. Even with realistic abstractions, the best abstraction may not exist. Moreover, the approximate fixpoint transfer theorem needs the monotony of all abstract transfer functions. On the plus side, the characterization of soundness using α (α • f • γ f ) defines a lower bound on f . Thus, the best f possible is this lower bound. We get a generic way to derive the best abstract transfer function from the concrete transfer function and the abstraction relation.

Concretization Framework

A related approach is to only use a concretization map. In this case, the concretization explains the concrete meaning of an abstract element. By dropping the abstraction map, we only have a way to check whether an abstraction is sound, but no generic way to deduce the abstract transfer function.

In this framework, we have an abstract complete lattice (A, ) and a concrete complete lattice (C, ). We also have a monotone map γ : A → C to interpret abstract elements. The notion of soundness of definition 11.14 "Soundness" (page 147) still stands. Galois connections enforce a relation between the order of abstract and concrete lattices. Of course, even without abstraction function, we need to have some constraints as well, in particular: is a sound approximation of ∧.

Fixpoint transfer theorems (theorem 11.2 "Exact fixpoint transfer" (page 148) and theorem 11.3 "Approximate fixpoint transfer" (page 148)) are not applicable here, we need another one.

Theorem 11.4 -Fixpoint approximation

Let (A, ) and (C, ) be complete lattices, and γ : A → C a monotone map. Let

f : C → C a monotone map, f : A → A and a ∈ A. If -f • γ γ • f : f is a sound approximation f , CHAPTER 11. ABSTRACT INTERPRETATION THEORY -f (a) a: a is a post-fixpoint of f , then a is a sound approximation of lfp f , i.e. lfp f γ(a) Proof. We assumed, f (a) a. f (a) a ⇒ γ(f (a)) γ(a)
by monotony of γ

⇒ f (γ(a)) γ(a)
since f is a sound approximation of f and by transitivity of

The proof of the lemma of Tarski's theorem (lemma 11.1) states that

lfp f = {x ∈ C | f (x) x} Thus lfp f γ(a)
Of course, this theorem is still valid in the context of Galois connections, but this context introduces more hypotheses (and thus, constraints) that are useless in this case.

We can note that the monotony requirement on the concrete transfer function is not prohibitive: usually, concrete transfer functions map set of states to set of states, using a bigger set of preconditions may only returns a bigger set of postconditions.

We can also check that the soundness condition is compositional. If we have

S 0 •γ γ • S 0 and S 1 • γ γ • S 1 . S 1 • S 0 • γ S 1 • γ • S 0 by soundness of S 0 and monotony of S 1 γ • S 1 • S 0 by soundness of S 1
There is no monotony requirements on abstract transfer functions anymore.

Here, the difficulty is to get a post-fixpoint of f in a constructive and automatic way.

Definition 11.15 -CPO (complete partial order)

A poset E = (E, ) is a CPO (complete partial order) if

∀ chain C ⊆ E, E C exists Proposition 11.14
Every complete lattice is a CPO.

Proof. In a complete lattice E = (E, ), for all X ⊆ E, E X exists. Thus, this is true for chains, in particular. Proof. By hypothesis a f (a). Since f is monotone, it preserves the order, thus

∀n ∈ N, f n (a) f n+1 (a) hence ∀(m, n) ∈ N 2 , m n ⇒ f m (a) f n (a)
Thus, I := {f n (a) | n ∈ N} is a chain and, since E is a CPO, E I exists.

f

( E I) = E f n+1 (a) n ∈ N since f is continuous = E E f n+1 (a) n ∈ N , a since ∀n ∈ N, a f n+1 (a) = E I So E I ∈ fp f . Moreover, ∀x ∈ fp f , a x ⇒ (∀n ∈ N, f n (a) x). So E I = lfp a f
This theorem has a stronger hypothesis than Tarski's, but it provides a constructive way to get the fixpoint. Sadly, this requires countable iteration (so infinite in general). But, we can get a sound approximation of the limit with finite iteration. For that, we need an over-approximation of the union that guarantees convergence in finite number of steps.

Definition 11.16 -Widening

Let (C, ) be a concrete complete lattice, (A, ) an abstract poset and γ : A → C the concretization map. A widening operator over A is a binary function ∇ : A 2 → A such that:

-∀(x, y) ∈ A 2 , γ(x) ∨ γ(y) γ(x∇y) (over-approximation of ∨)

-for all sequence (a i ) i∈N ∈ A N of abstract elements, the sequence

(b i ) i∈N ∈ A N defined by b i := a 0 if i = 0 b i-1 ∇a i otherwise is stationary.
In fact, in general, a widening works on sequence prefixes and previous result, but in practice, they just work with the last element of each prefix, making the widening a binary function.

Theorem 11.6 -Widening-based fixpoint approximation

Let (C, ) be a concrete complete lattice, (A, ) an abstract poset, γ : A → C the concretization map and ∇ a widening operator on A. Let f : C → C be a monotone map and f : A → A be a sound approximation of C. We define the sequence (a i ) i∈N ∈ A N by

a i := ⊥ if i = 0 a i-1 ∇f (a i-1 ) otherwise then
-a is stationary, and we denote l its limit, -lfp f γ(l).

Proof. From the definition of widening, it is very direct that a is stationary. We let

N ∈ N such that ∀n ∈ N, n N ⇒ a n = l. We have γ(l) γ(l) ∨ γ f (l)
by definition of ∨. Since the widening is a sound approximation of ∨, we have

γ(l) ∨ γ f (l) γ l∇f (l)
And since l is the limit of a, we have l = l∇f (l), thus, γ(l) = γ l∇f (l) . By transitivity of :

γ(l) γ(l) ∨ γ f (l) γ l∇f (l) = γ(l)
and by antisymmetry γ(l) = γ(l) ∨ γ f (l) , thus γ f (l) γ(l). By soundness of f , we have

f (γ(l)) γ f (l)
and by transitivity f (γ(l)) γ(l)

that is γ(l) is a post-fixpoint of f . Thanks to Tarski's theorem's lemma, we know that f indeed have a least fixpoint, and

lfp f = {x ∈ C | f (x) x} Thus, lfp f γ(l)
Here, we used a very permissive definition of the widening: it is a sound approximation of the concrete ∨. The drawback is that l is not necessarily an abstract post-fixpoint. We may strengthen the definition of the widening by replacing the condition

∀(x, y) ∈ A 2 , γ(x) ∨ γ(y) γ(x∇y) by ∀(x, y) ∈ A 2 , x y x∇y
which implies that we have an abstract lattice (and not simply a poset). Since is a sound approximation of ∨, by transitivity, this version implies the definition 11. 16. Moreover, we can prove directly that l is an abstract post-fixpoint, and we obtain the result by using the theorem 11.4 "Fixpoint approximation" (page 149).

We can remark that the proof of theorem 11.4 uses the fact that l is an abstract postfixpoint only to prove that γ(l) is a concrete post-fixpoint; so it may look dubious to use a stronger hypothesis to make the same proof, with additional steps. In the context we presented here, having an abstract post-fixpoint is indeed not very interesting, but it is if we want to check the result of an analysis: we cannot do much from the only knowledge that γ(l) is a concrete post-fixpoint, but we can check that l is an abstract post-fixpoint without knowing anything about the concrete world, or the widening.

To ensure termination, widening may lead to a loss of precision. To refine our approximation, there is a similar notion that is an over-approximation of the intersection and enforce convergence in finite time: narrowing operators. It is good to emphasize that narrowing operators are not dual widening: a widening must go past the least fix point, while a narrowing must stay above. A dual widening would go under the greatest fixpoint. Narrowing will not be detailed much more, as they are not relevant for the following.

Of course, being able to perform sound approximations of union, intersection and least fixpoint is mandatory, but the semantics contains probably more than that. Commonly, the semantics is expressed by composition of such operators and elementary transfer functions, e.g. assigning a variable, of enforcing a condition. For each transfer function, we need a corresponding sound approximation. The set of all these approximate transformers, with the explicative concretization, forms an abstract domain. The concretization is useful to justify the soundness of the analysis, but it does not appear in the implementation of an abstract domain.

Chapter 12

Product of Abstract Domains

A bstract domains only keep partial information about the state of the program.

Necessarily, the imprecise knowledge of the state may be too coarse to be able to check the interest properties.

Imprecision has two main sources. Firstly, abstract domains may be not as good as they can be. There are two possible reasons: abstract transfer functions may not be as precise as possible, or widening may be too brutal: by enforcing a very quick termination, we may have a very imprecise approximation of the least fixpoint. These are two kinds of loss of precision, but they can be solved by improving the domain, without changing the kind of property it handles. Depending on the case, we can improve abstract transfer functions, or use a more cautious widening whose termination will be slower, but approximation better. This kind of imprecision is common: usually, we do not implement abstract transfer functions optimally, but only well enough for existing cases. Likewise, widening is always heuristic, and depends on the application. Fortunately, improving a domain is usually easy. But sometimes, this is not enough: the domain is simply not suited to the problem.

The other kind of imprecision is more fundamental: an abstract domain is bound to a certain kind of properties. One may remember the range of variables or parity of variables, for instance. An abstract domain may also handle linear relations between variables, or modular equalities. We can abstract any kind of properties but, a priori, each domain takes care of one kind of property. When precision is lost between two statements, we may either consider more expressive properties that allows domains to keep their precision, or smash the two statements in a single call to a complex abstract transfer function. Indeed, the abstraction of the sequence of two statements can be more precise than the composition of the abstraction of each statement. For instance, in Listing 12.1, if we can only remember an interval of possible values for each variable, after line 4, z is in the interval [42, 52]. After line 5, we would get the interval [32, 52], thus we do not manage to prove the assertion. If we merge statements of lines 4 and 5, we can simplify them to z = y, which make the assertion easily provable. This method can work, but the number of transfer functions would explode, and the adequate grouping of statements may depend on domains. Thus, we prefer to handle more expressive properties, while keeping a constant instruction-wise granularity.

If the imprecision must be solved using a property that is not representable in the abstract domain we use, we need a new adequate domain for this kind of properties. In the example of Listing 12.1, we need to remember the linear equality z = y + x. Just adding an abstract domain do not solve this issue: they must cooperate. If they do not, from the point of view of the other domains, nothing changed and the same imprecision will happen. This chapter is about how we can make domains to cooperate. One can find further references in [CC79; CCF13].

Example: When Reduction is Needed

void f() { int v = 1; while(v <= 10) { v = v + 2; } if(v >= 12) { // error } }

Listing 12.2: A program that requires two domains

We consider the program on Listing 12.2. We assume we have a very common domain: the interval domain. This domain represents the value of each variable by an interval of possible values. We look at the values of v at the top of the loop (before testing the condition). At the first iteration, it is 1, 1 . Let us run the loop using only the union (without widening) to get most precision. We can do that here since it will stabilize in finite time. Another approach is to use a widening that will perform n unions at the beginning before returning Z, and we choose n big enough.

After one iteration, v has value 3; the value at the top of the loop is now 1, 3 . All Thus, it seems possible to enters the body of the if-statement, which is an error. However, since v is always odd, the value at the end of the loop is in fact only 11, 11 , but interval domain is unable to handle holes in intervals.

We need another domain: parity domain. This domain remembers whether variables are even, odd, or might be both. Let us run the example with this domain. Initially, v is odd. In the loop, v stays odd: fixpoint is reached in only 1 iteration. Before the line 6, v can only be odd. This analysis still detect wrongly the error since some odd integers are greater than 12.

If we make an analysis using both domains, before the line 6, we know that v ∈ 11, 12 ∧ v ≡ 1 [2]. Without cooperation, we would make the same mistake as before, and we would get v ∈ 12, 12 ∧ v ≡ 1 [2] inside the body of the if. To improve the precision and avoid the error, we need to be able to use each abstract property to refine the other. In this case, we can see that v ∈ 11, 11 ∧ v ≡ 1 [2] has exactly the same concretization as v ∈ 11, 12 ∧ v ≡ 1 [2]. But this improved version will not enter the if-statement. Similarly, we could refine v ∈ 12, 12 ∧ v ≡ 1 [2] into v ∈ ∅ ∧ v ∈ ⊥ parity since both have an empty concretization.

In this case, we could argue that a first parity analysis would be enough to know that v is odd, and it could feed the interval analysis with this property to get a precise analysis. But this method does not work if the parity domain needs information from the parity domain. For instance, if we replace the constant 2 by the expression rand(10)/1000 + 2 (which is equal to 2), the parity domain will not be able to understand by its own that v is odd: it needs the interval domain to simplify rand(10)/1000 into 0. The illwiller could argue that this problem can be solved by performing alternating interval and parity analysis, but that would just give extra work to communicate information between domains.

Reduced Product

Before improving abstract states, we have to make composite domains from two individual domains.

Proposition 12.1 -Product of complete lattices

Let (E 1 , 1 ) and (E 2 , 2 ) be complete lattices. Let E := E 1 × E 2 and be the relation on E defined by

∀((x 1 , x 2 ), (y 1 , y 2 )) ∈ E 2 , (x 1 , x 2 ) (y 1 , y 2 ) :⇔ x 1 1 y 1 ∧ x 2 2 y 2 (E, ) is a complete lattice.
Proof. We can indeed check the minimum in E is (⊥ 1 , ⊥ 2 ) and the maximum is ( 1 , 2 ). Similarly, least upper bound and greatest lower bounds are applied pointwise: let

A ⊆ E A := ( 1 {x ∈ E 1 | (x, _) ∈ A}, 2 {y ∈ E 2 | (_, y) ∈ A}) A := ( 1 {x ∈ E 1 | (x, _) ∈ A}, 2 {y ∈ E 2 | (_, y) ∈ A})
A similar result holds for Galois connections.

Proposition 12.2 -Product of Galois connections

Let (C, )

γ 1 ← ----→ α 1 (A 1 , 1 ) and (C, ) γ 2 ← ----→ α 2 (A 2 , 2 ). Let A := A 1 × A 2 , γ := (x, y) ∈ A → γ 1 (x) ∧ γ 2 (y), α := c ∈ C → (α 1 (c), α 2 (c))
and be the relation on A defined by

∀((x 1 , x 2 ), (y 1 , y 2 )) ∈ A 2 , (x 1 , x 2 ) (y 1 , y 2 ) :⇔ x 1 1 y 1 ∧ x 2 2 y 2 We have (C, ) γ ← ----→ α (A, ).
Proof. Obviously, α and γ are monotone. Since α is monotone, and α 1 • γ and α 1 • γ are contractive, α • γ is contractive as well.

Similarly, γ • α is extensive.

Using the alternate definition of Galois connections (proposition 11.11 "Alternative definition of Galois connections" (page 146)), we have the result.

It is interesting to remark that the concretization is the intersection of both concretization maps. Indeed, since both domains are sound, the concrete state is lower than the concretization of both abstract states, thus, it is lower than the intersection. We can also understand it in terms of properties: if two properties hold, the conjunction is true as well.

PARTIALLY REDUCED PRODUCT

159

This gives a way to improve the abstract states using each other. In the context of the previous proposition, let us define

ρ : A → A (x, y) → (α 1 (γ 1 (x) ∧ γ 2 (y)), α 2 (γ 1 (x) ∧ γ 2 (y)))
that is, ρ = α • γ. The concretization allows each domain to contribute to the overall precision, and by abstracting again, we gain a more precise abstract state. We indeed know that α • γ is contractive. This is the best possible improvement. Moreover, we can easily check that it is still sound, i.e.:

∀a ∈ A, γ(a) γ • ρ(a)
We could also say that ρ is a sound approximation of the identity. Indeed, since

(C, ) γ ← ----→ α (A, ), we have γ • ρ(a) = γ • α • γ(a)
by definition of ρ = γ(a) by proposition 11.12 "Iteration of adjoints" (page 147)

Which is even stronger, but it is not the point here. However, this reduction operator is not usable in practice. First, we need to use the Galois connection framework. We explained in section 11.2 "Galois Connections Framework" (page 145) why this framework is usually too strong to be usable in practice, in particular, the abstraction map does not always exist. Moreover, even with an abstraction map, the concretization is often very expensive to compute in general, if it terminates. As a consequence, this direct definition is not usable in an implementation. An alternative approach is not to go by the concrete world but compute, by hand, the reduction operator, thanks to this definition, and implement the resulting operator. This method may seem appealing but fails in general: the reduction operator may still be very expensive to compute, and we need to compute it for all combinations of abstract domains. Moreover, we still need the theoretical framework of Galois connections.

We need another way to perform reduction. It might be an approximate reduction but it has to be efficiently computable and implementable modularly.

Partially Reduced product

Usually, fully reduced product is not necessary, but we need an effective reduction method and, to be able to implement it in practice, we need it to be modular: adding a domain should not require to update all existing domains.

Definition 12.1 -Sound partial reduction

Let D be the concrete domain and D 1 and D 2 be abstract lattices with respective concretization γ 1 :

D 1 → D and γ 2 : D 2 → D. ρ : D 1 × D 2 → D 1 × D 2 is a sound reduction operator if ∀(a 1 , a 2 ) ∈ D 1 × D 2 , let (b 1 , b 2 ) := ρ(a 1 , a 2 ) in γ 1 (a 1 ) ∩ γ 2 (a 2 ) ⊆ γ 1 (b 1 ) ∩ γ 2 (b 2 )
Morally, we want to improve a 1 and a 2 , so in most cases, b 1 1 a 1 and b 2 2 a 2 . But this is not a requirement and, indeed, it is not always verified in real-world domains.

The solution to make a real-world reduction operator we will describe here is explained in detail in [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF]. The point is to use an abstract lattice IO , called communication channel, common to all domains, that allows them to communicate. We are also given a concretization γ IO : IO → D where D is the concrete domain.

Each abstract domain D (with concretization γ) features two additional functions:

Extract : D × IO → IO Refine : D × IO → D
Of course, we need soundness requirements:

-∀a ∈ D , ∀io ∈ IO , γ(a) ∩ γ IO (io) ⊆ γ IO (Extract(a, io)) -∀a ∈ D , ∀io ∈ IO , γ(a) ∩ γ IO (io) ⊆ γ(Refine(a, io
)) Extract function enriches the content of a communication channel given an abstract element. Refine uses the content of a communication channel to improve an abstract element.

The communication channel must be able to represent all properties that are inferred in a domain but used in another one. It is a conjunction of a lot of properties, but there is no transfer function in the communication channel. Adding a new kind of property does not imply more work, especially, all Extract and Refine functions are unchanged: they respectively never emit such property and ignore it. We just need to update the Extract function of the domain that deduces the interesting property and the Refine function of the domain that uses it.

The way we achieve that is merely technical and very dependent on the language we are using. We will simply detail how it is done in Astrée as an example to demonstrate that this is perfectly feasible.

In Astrée, we use in fact two communication channels according to the kind of communication we need. This implies that each domain implements two versions of Extract and Refine.

Astrée uses a product of a lot of domains; the product orders them, and their abstract transfer functions are evaluated in this order. The first channel follows this evaluation order, it is called input channel. Each domain receives the state of the communication channel corresponding to the precondition and the postcondition, and can refine the postcondition. The constraints on the precondition come from the postcondition of the previous statement and the postcondition comes from IO that has been refined by domains that have been run already. With this communication channel, information only flows from left to right, as domains are used. Once all domains have been run, the postcondition is refined by all domains and used as the precondition for the next statement.

Input channel is implemented as a product type, each field storing some kind of property. Domains read only fields they can use and update fields they can refine. If we need to add a new field, we just have to update the type definition and the initial value IO . Domains, by default, will simply ignore this field and never refine it, which is safe. To implement a reduction, we need to make the sender domain to refine this new field, and the receiver domain to read it and refine its own internal state accordingly. In this kind of communication, each domain communicates information without knowing if it will be used. It may seem uselessly costly, but we will see in subsection 13.2.2 "The Tree of Relational Numerical Domains" (page 167) that memoization and laziness make unused communication almost free.

The other kind of channel is more active: domains are allowed to emit some constraints they judge to be especially interesting. This channel, called output channel, allows two kinds of communication: to all domains (broadcast channel) or only to underlying domains (domains on the left, those that have been run before). Each concerned domain refines its internal state using the constraint. This communication channel allows information to be sent back to first (leftmost) domains.

Output channel is implemented using a sum type, each constructor representing some kind of property. Domains emit only constructors corresponding to properties they can deduce, which may be none: domains are not required to emit anything in the output channel. However, domains must be able to refine their internal state using such constraint. Unknown (or irrelevant) properties are ignored, which is safe. To enrich the output channel, we just need to update the type definition: new constructors are ignored by default by a catch-all pattern. To add a new reduction, once again, we just need to update the sender and the receiver.

This way of building reduction operators is easily implementable in practice, can be made efficient and precise enough. This is a very parametric framework. It has another benefit, but theoretical: just as implementation is distributed across domains, soundness proofs are also distributed. Indeed, we simply need to prove the soundness of Extract and Refine functions of each domain to get the soundness of reduction operator.

Yet, we must take care of termination: enforcing a constraint may generate new constraints, causing new reductions and so on. We cannot simply iterate reduction naively. Simple strategies solve this issue: like setting a maximum number of iterations or performing reduction only in the directions allowed by a directed acyclic graph.

Chapter 13

Astrée

M ost of the work presented in following parts has been implemented in a develop- ment version of Astrée. Astrée [Cou+01; Bla+03] is a static analyzer designed to analyze critical C source code coming from automotive, avionics, astronautics, etc.

Astrée stands for "Analyseur statique de logiciels temps-réel embarqués" (real-time embedded software static analyzer). The development of Astrée started in November 2001 and is still going. Due to its applications, Astrée has some limitations like a limited support of dynamic allocations or the absence of support of variable-length arrays (VLAs). Astrée is based on abstract interpretation and is sound: it finds all possible errors. It has some industrial successes like the proof of absence of RTEs in the primary flight control software of Airbus A340 and A380 fly-by-wire system and in the automatic docking software of the ATV * , the cargo spacecraft developed by the ESA † to resupply the ISS ‡ (see [START_REF] Cousot | The Astrée Static Analyzer[END_REF]).

Astrée has been extended in AstréeA [Cou+06a] (which stands for "Analyseur statique de logiciels temps-réel asynchrones embarqués" (real-time asynchronous embedded software static analyzer)). The main improvement of AstréeA is the capacity to analyze multi-threaded software. The name Astrée is both used for the historic version and the family, in particular, nowadays Astrée usually stands for AstréeA, which is the last released major version. Marketed by AbsInt § , Astrée has now many industrial applications.

The changes explained in the following parts are implemented in a development version of Astrée, called AstréeS (Astrée for Security). AstréeS is developed on top of AstréeA and may become the next major version of Astrée family.

In the following, we will detail some implementation aspects of the work presented hereafter. To introduce the context required for these remarks, in this chapter, we present the architecture and some implementation details of Astrée.

Overview of the Architecture

Astrée is written mostly in OCaml [Ler+20], with a few parts in C. Astrée makes an intensive usage of OCaml modules and functors. First, let us describe the global architecture of Astrée.

Astrée includes its own C parser that is able to parse custom analysis directives that are included in source code thanks to specific syntax extensions. Parser represents C programs in an abstract syntax tree (AST) of a format called C. This format is very syntactic: typing is not checked yet, redundant constructions are neither simplified nor unified, even parenthesis in arithmetic expressions are still mentioned in this AST.

The C tree is not directly usable. It is translated to another AST called Cc (or Cc_code). This tree is much more elaborated: it contains types at all expression nodes and operators. Variables and functions are identified by integer unique identifiers (UID) to solve explicitly scoping. Some syntactic sugar is removed, but we keep as much syntactic information as possible ¶ . For instance, Cc have only one node for loops, but with optional fields to represent while-loops as well as for-loops without transforming them into a while-loop. This is because syntax carries some good hints about the semantics that can be very useful for heuristics. For instance, in a for-loop, it is usually very easy to find the variable that acts as loop index and adapt the abstraction accordingly. It would be clumsy to compile for-loops to while-loops and trying to decompile while-loops to get information that was explicit in the first place.

The Cc AST is not processed enough to be easily used. It is translated a last time to another format called Graph. In contrast to what the name may indicate, it is not a CFG, but still an AST with horizontal links, for instance between goto-statements and labels, because they are not syntactically linked; yet, control flow nodes that are more local (like loops or conditional branching) are still represented as subtrees. This format contains almost all the information that may be computed statically and that is useful during the analysis. For instance, goto-statements are annotated with the set of variables to remove and to add. Another interesting point is that expressions are function callfree. Function calls are represented by a separate node, just as we did in Asclepius (see chapter 4 "A Minimal C-like Language" (page 43), and especially definition 4.3 "Statements" (page 44)). Indeed, when function calls are ordinary expressions, the semantics is more difficult to define since function calls imply running statements. Graph AST also includes synthesized code, in particular "glue" code added before and after function calls to initialize parameters variables (to the expressions in arguments) and to copy the returned value in the variable from the function call. Graph tree is the format on which we run the analysis.

Other things need to be done initially. A very important one is to instantiate abstract domains according to configuration (coming from files and command line). Astrée uses a composite abstract domain made of a stack of parametric domains (including a product of domains). This is implemented with OCaml functors and will be detailed in section 13.2 "Composite Abstract Domain" (page 165).

Astrée instantiates also a central component: the iterator. This is the module that iterates over the Graph tree and calls appropriate transfer functions of the abstract domains. It will be detailed in section 13.3 "The Iterator" (page 169). Morally, this is the only part that is language-dependent. Abstract domains should be seamlessly pluggable into another interpreter designed for another language. Of course, the separation is not that strict, especially because the transfer functions we need in domains depend on the features of the language. We can compile any language to use a minimal set of transfer functions, but this would mean to give up information that is obvious in the first place. This is the same kind of trade-off that we discussed when examining the question of loops. We can have a lot of transfer functions, some of them may be complex, while keeping most original structure and information. The other strategy is to compile to a reduced set of transfer functions, and we can expect them to be simpler. In this last case, to get syntactic information, we need to reverse this compilation, which is not robust. In most cases, in Astrée, the chosen strategy is to keep the analyzed program as it is, even if it requires more complex transfer functions.

Composite Abstract Domain

Astrée uses a composite abstract domain made of a big tree of parametric abstract domains. A parametric abstract domain is a domain that is defined with respect to other domains. It can be seen as a function whose parameters and images are abstract domains. Of course, there are non-parametric domains as leaves of the tree of domains.

They are several uses to parametric domains. They can simply be reduced product, as presented in section 12.3 "Partially Reduced product" (page 159) (and [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF]), or the improved product explained in Part IV "A Reduced Product with Ghost Variables" (page 229). Indeed, given two domains the reduced product builder produces a new domain that have the expressive power of both domains. By iterating, we can generalize this product to take an arbitrary number of domains to combine.

Another application of parametric domains is to change the type of abstraction. For instance, from a domain that abstract sets of integers, we can build a domain that abstracts memory states by mapping each variable to an abstract state of the underlying domain. This abstraction is non-relational: each variable is abstracted independently of each other.

Astrée uses these two kinds of parametric domains. On the top (i.e. closer to the interpreter, the outermost domains), there is a stack of domains that abstract away several kinds of language features. On the very top, they abstract executions traces with concurrency, while the domain at the bottom only abstracts numbers. Consequently, each domain has its own signature (module type) adapted to the type of the concrete domain. This stack of domain is thus very rigid: they cannot commute since we need to simplify the type of the concrete domain from the outside to the inside.

Under this stack of domains, there is a tree of reduced product of relational numerical domains. At this level, pointers have already been simplified, domains only see physical memory cells. These domains abstract set of maps from memory cells to integers of floating-point numbers. The leftmost domain of this tree is particular: it is a nonrelational domain build from a domain that abstracts set of integers or floats, as explained previously. In fact, this underlying domain is also composite: it is a product of domains that abstract numbers, like the interval or the parity domains used in the example of section 12.1 "Example: When Reduction is Needed" (page 156).

Let detail each part from the bottom.

The Tree of Non-relational Numerical Domains

At the very bottom of the tree, there are two products of non-relational domains: one abstracts integers and the other abstracts floating-point values. These domains are very simple and their interface is as well. For instance, they cannot evaluate directly any arbitrary expression, but they provide transfer functions for elementary operations only. They are like abstract pocket calculators.

Their reduction possibilities are also limited. Transfer functions do not directly take communication channel on the pre-and postcondition. Here, communication is made separately.

Both trees (integers and floats) are lifted to a domain that abstract functions from memory cells to values. Memory cells are typed, thus, we know which should map to integer abstract value and which should map to floats. The lifter domain fulfill other tasks. Especially, it manages iteration on underlying domains and communication between them.

From a theoretical point of view, the non-relational composite domain abstracts an element of P (C → V ) (where C are memory cells and V the set of possible values) into an element of C → P (V ). For instance

a → 1 b → 1 , a → 2 b → 2 is abstracted into a → {1, 2} b → {1, 2}
losing the relation a = b. Indeed, its concretization is

a → 1 b → 1 , a → 2 b → 2 , a → 1 b → 2 , a → 2 b → 1
Then, the abstraction of P (V ) is delegated to underlying domains. Likewise, the implementation of such a domain usually works with an associative container from C to elements of the underlying abstract domain.

This composite domain is very imprecise, as it is unable to establish relations between variables, but on the other hand, it is very efficient both in time and memory consumption. Moreover, this domain is able to run, even imprecisely, any kind of expression: it has an opinion on everything, even if it is not an accurate one.

The Tree of Relational Numerical Domains

A non-relational domain is a special case of relational domain. From the non-relational composite domain, and other (actually) relational numerical domains, we build a product of domains that abstracts elements of P (C → V ). This product implements more finely communication between domains, and the signature of these domains is much richer than non-relational domains. Non-relational domains define functions only for elementary arithmetic and logical operators. On the opposite, relational domains directly run statements, like assignments. This allows domains to evaluate expressions by induction according to their own abstract semantics, or to pattern-match expressions to detect expressions of a certain form (e.g. linear combinations) for specific treatments.

Input channels are implemented as a product type of functions. Each field answers a single kind of question, for instance, one of them tells if two variables are equal: it takes two parameters (variable identifiers) and returns a Boolean value. Of course, precomputing all possibilities for each field may be very expensive, and most of them are never used, thus fields implement lazy computation with transparent memoization, so only used properties are computed, and only once. This way, unused properties have a negligible cost.

The downside of using functional values is that they are hard to debug. Indeed, when there is a bug in such a function, it should be understood in the context where this function was created, and not in the context of execution (when the bug appears). But when the bug appears, the context of creation of the function does not exist anymore. This observation will explain several choices in the following.

The Stack of Domains

The relational numerical domain is not sufficient to analyze C programs. This domain is thus gradually adapted to a domain that is able to take all aspects of C into account by a stack of parametric domains. The resulting composite domain is drawn on Figure 13.1, and domains of the stacks are detailed below, from bottom to top. Real-world programs are often multi-threaded. The parallel domain allows analysis of multi-threaded programs using a rely-guarantee method.

Underlying domains have the responsibility to track the effect of assignments and to apply the effect of assignments performed by other threads. Nevertheless, except these operations, that are explicit, underlying domains run the program as if it was in a single thread, so without worrying about multi-threading. One can find details about this domain in [Min13; Min14]. 4. Trace domain. Trace domain allows state partitioning according to an automaton.

We will not be interested anymore in this one. 5. Environment domain. Under this domain, execution traces seem uninterrupted.

To handle discontinuity in control flow (like goto, continue, break or return statements), environment domain implements unsynchronized iterations (see [START_REF] Cousot | Méthodes itératives de construction et d'approximation de points fixes d'opérateurs monotones sur un treillis, analyse sémantique des programmes (Iterative methods for construction and approximation of fixpoints of monotone operators on lattices, program static analysis[END_REF]). This domain receives commands that describe the control flow. For instance when it receives the command corresponding to a goto, it suspends the current execution, and restores it only when it gets the command signaling the corresponding label. The way this domain works is explained more precisely in section 17.2 "Intra-function Near Jumps" (page 212), as we extend it to handle another kind 13.3. THE ITERATOR 169 of jumps using a method very similar to what is already implemented. 6. Partitioning domain. This domain performs partitioning according to the trace prefix: whether it went through an if, or through the else, for instance. This part is not very interesting to us. Nevertheless, as it is just under the iterator, it has special duties. In particular, the iterator prepares the function that performs the analysis of a C function and gives it to the partitioning domain to stabilize backward gotos and handle return statements.

The Iterator

The composite domain that we have described defines the type of abstract values and transfer functions to act on abstract values. The iterator is in charge of reading the program and calling transfer functions accordingly.

As input, the interpreter takes a program in the Graph format already mentioned. We remind that this format is mainly an AST, and not a CFG. Iterating on a CFG is simpler: one just has to identify loops, choose at least one widening node on each loop (though this decision can be made dynamically) and use a worklist algorithm. This algorithm keeps a list of nodes to update. At each step, it takes a node, updates it according to its predecessors and if the condition is not stable, it adds the successors in the list. This algorithm makes the abstract execution (the analysis) very different from the concrete execution, and thus bugs of the iterator and domains may be difficult to find and understand.

Moreover, transforming the program into a CFG drops most syntactic information, that could help heuristics. There is another drawback with CFGs: we need to be able to build it, making languages with dynamic jumps more difficult to handle. Even if pure C does not have such dynamic jumps * * , x86 has (and consequently, mixed C and x86 also). It is relatively easy to dynamically compute successors of a node, but we need to dynamically chose widening-nodes as well. This is a uselessly complicated design.

In contrast, the AST approach has the main drawback to force the iterator to handle a tree with a large variety of nodes, one for each language feature. It may also require more transfer functions in the domains. Once again, it is a reasonable trade-off since it allows domains to be more precise than they would be by decomposing complex operations into elementary ones.

The AST approach is very similar to the inductive definition of a denotational semantics. There is one tricky feature of C that is not obvious to handle: function pointers. When a function is called through a pointer, we need to get the functions this pointer can designate. For this purpose, the abstract value of the pointer must be concretized. Usually, we avoid this operation as it can be costly and the concretization of an abstract value can be a very large set, even infinite. Here, this is more reasonable: the concretization cannot be bigger than the set of all functions of the program, which is usually not excessive. Then, we check there is no recursive calls. Astrée does not handle recursivity: it is both very difficult to do precisely and useless since Astrée is mainly dedicated to embedded programs, which do not allow recursivity. To detect recursive calls, the iterator relies on the partitioning domain which keeps information about past events, such as function calls; if the same function is already present in the trace, the call is recursive and an alarm is raised. Yet, the analysis continues normally for functions in the concretization of the pointer that are not recursive calls. At the end of the analysis of each possible function, postconditions are joined.

This strategy is very precise, as it inlines all function calls, but might be costly in computation time. However, a weaker context-sensitivity is not sufficient for industrial applications. Moreover, it has a serious consequence: the body of a function is analyzed only if it is called. As it seems natural, it hides the fact that it forbids C longjmp. As this function is close to an assembly jump between C functions, and this case actually exists in industrial code, we will need to adapt the iterator accordingly.

Chapter 14

Abstraction of Arithmetic and Logic Statements

A bstracting assembly statements requires an enriched model compared to C. Yet, additions and modifications are not trivial and motivated by the semantics of assembly instructions. For these reasons, to make the explanations clearer, we will not detail the model, then the abstract semantics etc. Rather, we will show how to abstract statements by category, introducing elements of the model progressively.

We start by showing the abstraction of arithmetic and logic statements, as they are the simplest. Especially, this class of statements is simple enough so that we do not really need to detail the abstract transfer functions, but we show the consequence they have on the modeling.

Model

Assembly instructions, unlike C statements, can access registers. We need to abstract them in such a way that their values are persistent between instructions. Otherwise, any write to a register would fall into oblivion and reading it would give the most general value . Abstracting general-purpose registers as C int (or unsigned int) variables is, of course, not wrong, but it is not sufficient either. This question will be explored in the next section.

We will in particular emphasize which parts of the central processing unit (CPU) we are ignoring.

A very standard feature of processors is the cache. The need for caches stands in two remarks: the main memory is slow to access from the CPU and programs I/O usually are concentrated on small regions of memory. A cache is a piece of memory of low capacity * , but very fast that acts as a proxy for the memory: when a zone in memory is read, it is loaded in cache. Following reads and writes will just query and update the cache. 174CHAPTER 14. ABSTRACTION OF ARITHMETIC AND LOGIC STATEMENTS When asked explicitly (e.g. with WBINVD instruction), or when the cache is needed for something else, modified cells in cache are written in memory.

Nowadays, processors have multiple cores and multiple levels of caches: fastest caches are smallest. Some caches (biggest and slowest ones) are usually shared between cores, but fastest caches are specific to each core. This is a serious matter since writes to a non-shared cache will not be visible from other cores. Writebacks will occur later and might be overwritten by another core that is not aware of modifications done by all other cores. This kind of memory model is called weakly consistent (see [START_REF] Dubois | Memory Access Buffering in Multiprocessors[END_REF]). The study of such memory models is not the purpose of our work. Nevertheless, we trust that taking such considerations into account will not invalidate the following. We think it might be solved modularly by wrapping memory domains with a parametric domain (implemented as a functor) that lifts an abstraction of consistent memory to an abstraction of weakly consistent memory, just like the domain described in [Min14; Min13] lifts an abstraction of sequential programs to an abstraction handling multithreading.

When memory ranges used by each core are disjoint, caches are transparent, i.e. memory accesses behave like there were no cache. An even more special case is when there is only one core (or at least, we use only one). Our target software indeed uses only one core, thus it is safe to ignore caches and related instructions.

Caches are managed according to cache policy, but this is not documented: we do not know how the processor uses the cache. For example, when it needs some space, it may invalidate the oldest, the less recently accessed or the less often used data. All are legitimate strategies and have very different behaviors. The CPU might also use a hybrid strategy, so that, trying to guess is hopeless.

In addition to caches, processors have other internal hidden memories. We already mentioned TLBs (see subsubsection 2.3.3.4 "Translation Lookaside Buffers" (page 28)), that are a special kind of cache to speedup address translation for memory accesses. TLBs have a mixed management: they are mainly managed by the processor, but the user can influence the behavior by marking some entries as persistent, or, conversely, to invalidate them precociously. Since they may be manually managed, TLBs may have a semantic impact if they are poorly used. Indeed, if a page is marked as persistent though it belongs to a simple user process, it will not be purged when a task switch occurs, and address translation will be faulty. Nevertheless, TLBs are ignored for now since our study case does not use them explicitly. Moreover, they are a part of paging, which is not handled yet.

There are other caches that allow the processor to make smart guesses for jumps. Indeed, instructions are processed in several stages: when an instruction enters the second stage, the first one is already free for the next instruction. When we run a conditional jump statement, we need to guess whether the condition will be true, to choose the next instruction we start to execute, before actually being able to decide. If the guess is correct, the execution is pursued, otherwise, the current instruction is dropped, and the right instruction is run instead. This technique is called speculative execution. To help this guess, the processor uses a memory of lastly evaluated conditional jumps. This kind of additional memory is ignored in our abstraction as it does not impact 14.2. REGISTER ABSTRACTION the semantics of instructions.

A very important kind of hidden memory is the hidden parts of registers (see subsubsubsection 2.3.2.2.4 "Segment Selectors" (page 23) and subsection 2.5.2 "Task Register" (page 35)). They are not accessible by the software, but they are documented and crucial for the semantics. For instance, the hidden part of a segment register is loaded from the GDT at the same time as the segment register itself, then only the hidden part is used during address translation, in order to minimize accesses to the GDT that is in (slow) memory. A side effect is that if the GDT is modified but the segment register is not reloaded, it has no effect on segmentation. For that reason, we must take hidden part of segment registers into account. This will be detailed in section 16.1 "System Statements" (page 195).

There are probably other undocumented caches, Intel processors are famous for that. They are not documented because they do not impact the semantics with respect to the model described in [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF]. They have an impact that is critical for other kinds of studies, especially, when we are concerned about execution time, as well as when we are seeking a bound or when we are interested in determinism. Since we do not care about timing, we can ignore these kinds of undocumented features.

To sum up, the main addition with respect to C is the presence of registers. Another important difference is the point of view over the stack. While C and assembly both work with the same stack they consider it with a different level of abstraction. This will be treated in a following chapter (see chapter 15 "Stack Abstraction" (page 183)).

Register Abstraction

General-purpose registers are type-agnostic sequences of 32 bits: they can contain a signed integer, an unsigned integer, a binary coded decimal number, a pointer or simply a sequence of flags. Unlike C variables, that are considered as numbers independently of their representations, the meaning of assembly register depends on the context. The problematic point is not about storing arbitrary values into a variable, but that C operators behaviors change according to the type. For instance, in the snippet of Listing 14.1, at line 2, y stores a pointer. The C standard allows this conversion up to some implementation defined behaviors that are known to be favorable in our case. Astrée is already able to analyze such code, thanks to the pointer domain (see [START_REF] Miné | Field-sensitive value analysis of embedded C programs with union types and pointer arithmetics[END_REF][START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF]). But the interesting point is that the increment of line 3 must be understood in integer arithmetic, and not in pointer arithmetic.

Due to type-agnosticism of registers, the meaning of each operation do not depend 176CHAPTER 14. ABSTRACTION OF ARITHMETIC AND LOGIC STATEMENTS on the type, but on the opcode. Some operations are meaningless with some types. For instance, applying a binary coded decimal instruction to a register storing a pointer will give an unpredictable result. This kind of situation is easy to detect, and we can raise an alarm and default to in such cases. A problem arises when a single opcode is valid for multiple datatypes that are compatible. For instance, ADD performs the sum as well for signed or unsigned integers, thanks to two's complement. It is not a priori known whether the result will be used as a signed or unsigned integer. We shall assume both are possible. Moreover, to set accurately the status flags in EFLAGS register (see subsubsection 2.2.1.2 "EFLAGS Register" (page 11)), we indeed need both signedness.

In our application, this is the main ambiguous case. Thus, we modeled each register by a pair of variables: a signed integer and an unsigned integer. This abstraction relies entirely on the underlying abstraction of variables, and its concretization is the intersection of two's complement representation in each concretization.

Formally, given an abstract domain D , if we let r s ∈ D (resp. r u ∈ D ) be the abstract value representing a signed (resp. unsigned) 32-bit integer that represents register r, γ s : D → P (Z) (resp. γ u : D → P (Z)) be the concretization of a signed (resp. unsigned) variable and β : Z → {0, 1} 32 the two's complement representation of an integer over 32-bit. The concretization of the register r is For instance, the code on Listing 14.2 copies a to b through registers EAX and EBX. Roughly, it is executed as code on Listing 14.3. The assignment from a to EAX is executed as an assignment from a to signed and unsigned values of EAX. We can notice that the unsigned version receives *((unsigned int*)(&a)), this is the binary sequence of a reinterpreted as an unsigned int. Then, when EAX is used, we select the version to use according to the type of the destination: a register destination requires two copies (like the copy from EAX to EBX), but a C variable destination allows a more precise decision based on the signedness of the destination. When the operation using a register is not signedness-agnostic, like multiplication, we select only the right version of the register to perform the operation. Both variables associated to the register are updated according to the result: one is a mere copy, the other requires to be reinterpreted.

γ : D → P {0, 1} 32 (r s , r u ) → {β(n) | n ∈ γ s (r s )} ∩ {β(n) | n ∈ γ u (r u )}

PARTITIONING ACCORDING TO EFLAGS

It is interesting to remark that we cannot directly rely on the corresponding C operation. For instance, overflow during C addition of signed integers is an undefined behavior, but it nicely wraps around in assembly. In C, we perform the integer addition, followed by a bound check, while in assembly, we use the integer addition followed by a modulo. The difference is quite simple, but it shows once again that a C stub for assembly instructions is insufficient.

Partitioning According to EFLAGS

This section has been co-thought with Yves-Stan Le Cornec, who had a research engineer position in Antique team at that time, and entirely implemented by him in the development version of Astrée.

Arithmetic operations set status flags in EFLAGS register. These flags are typically used by conditional jumps. Let us illustrate with Listing 14.4. The CMP instruction performs a subtraction and sets status flags accordingly but does not save the result. The conditional jump mnemonic JE (jump if equal) is an alias for the mnemonic JZ (jump if zero), indeed EAX -EBX = 0 ⇔ EAX = EBX. In detail, it jumps if ZF flag of EFLAGS register is set. In this example, the control jumps to label: if and only if EAX and EBX are 178CHAPTER 14. ABSTRACTION OF ARITHMETIC AND LOGIC STATEMENTS equal. A way to assign ZF is to compute the intersection of abstract values of EAX and EBX. If the intersection is not empty, then ZF may be set. Conversely, ZF may be cleared if EAX and EBX may have different values. Without further precaution, this is too naive, because the constraint ZF = 1 does not propagate backward to EAX = EBX. We need to remember that

(ZF = 0 ⇒ EAX = EBX) ∧ (ZF = 1 ⇒ EAX = EBX)
This kind of relational information is called partitioning. One can use any kind of condition to partition the abstract state, including a Boolean formula defined over status flags. Instructions that update status flags, e.g. arithmetic or logic instructions, perform such partitioning so that a possibly following conditional jump would behave correctly. This is implemented using binary trees [START_REF] Mauborgne | Representation of Sets of Trees for Abstract Interpretation[END_REF] where nodes are annotated with conditions and leaves are abstract states of the underlying domain.

Implementation

Internally, registers are declared as global variables. They are added into the set of global declarations, just before being passed to the interpreter for allocation.

The translation from source code instructions to instructions on signed and unsigned representation of registers must be done above the struct domain (as explained in subsection 13.2.3 "The Stack of Domains" (page 167)), since variables are translated into physical cells in this domain. There is no intrinsic upper bound for the position in the stack of domains, but it is better to put is as low as possible: the lowest it is, the most features are already abstracted and the simplest the domain is to implement. The new structure of the composite domain is shown on Figure 14.1: there is a new CPU domain that is in charge of translating instructions as they are in the source code to the instructions required for underlying domains. This domain is not limited to this task, as we will see later (see chapter 15 "Stack Abstraction" (page 183) and chapter 16 "Other Non-control Statements" (page 195)).

Envisaged Improvements

Several improvements have been considered; this section will detail some of them. Most of them have not been implemented since it was not felt useful, neither in terms of performance nor precision. Moreover, while some of these optimizations are very lightweight, others implies heavy modifications of the implementation.

Precision Improvements

The first kind of improvements aims to increase the precision of abstractions, so as to avoid false alarms.

Reduction of Both Register Representations

Signed and unsigned versions of registers are updated independently as much as possible. When only one signedness is available, it is reinterpreted so as to fill the other version. 180CHAPTER 14. ABSTRACTION OF ARITHMETIC AND LOGIC STATEMENTS This reduction was not implemented as it was not necessary for our applications. The next subsubsection describes another precision improvement that was enough by its own.

unsigned value signed value 2 N -1 -1 2 N -1 -1 -2 N -1 2 N -1 -1 2 N -1

Wrapped Interval Domain

Another approach to handle the signedness-agnosticism of registers is to directly use abstract property that are themselves signedness-agnostic. Some properties used on integer arithmetic can be quite nicely adapted to modular arithmetic. In particular, this is the case of the interval domain: we can think of modular intervals. Such a domain has been implemented as described in [START_REF] Navas | Signedness-Agnostic Program Analysis: Precise Integer Bounds for Low-Level Code[END_REF]. Yves-Stan Le Cornec implemented this domain in our development version of Astrée.

The idea of such domain is to allow intervals to wrap around, as illustrated on Figure 14.3. This is useful when the interval representing a variable is pushed (by an addition) on each side of the maximum value of the type. Using a simple (integer) interval domain, the convex hull of such a wrapped interval would give all representable values, which is the most imprecise abstract value. With a wrapped interval domain,

(-1 ≡ 2 n -1 [2 n ]) 1111 (-2 n-1 ≡ 2 n-1 [2 n ] ) 1000 0111 s = u -2 n s = u

Performance Optimization

In addition to making the analysis precise enough to prove that the code is correct, we are also interested into analysis time. Since we are targeting industrial applications, one must be able to run the analysis in reasonable time. This is the second kind of optimization: without losing precision, we want the analysis to run faster. In fact, it may be acceptable to have a less precise analysis as long as we can still prove desired properties.

Optimizations mentioned here should not impact precision, but only execution time. They have not been implemented since the performance of this part of the code is considered to be good enough, as far as we know.

Minimizing Partitioning

When there is a sequence of arithmetic instructions, status flags of each instruction are overwritten by the next one. One can thus avoid partitioning according to flags that are known to be ignored. For instance, in Listing 14.5, flags set by SUB of first line are always overwritten by ADD on the second line. Thus, it is useless to partition at the first line. The third line checks the value of OF flag, thus, we need to partition at least according to this flag at the second line. But is would be foolish to think that it is enough. Indeed, at the destination of JO, we may need the other flags, for instance, if the next instruction is another conditional jump. When destinations of jumps are statically known, we may build the dependency graph to find which flags of which instructions may be used. When a jump has a dynamic destination, this deduction is impossible, and we must consider that all flags may be used. This is morally simple, and in fact a classical method in compilation to find unused values. Implementation is also quite straightforward but a bit extensive as it would 182CHAPTER 14. ABSTRACTION OF ARITHMETIC AND LOGIC STATEMENTS sub EAX, EBX add ECX, EDX jo label Listing 14.5: A situation where some flags may be ignored require deducing more information in the front end, and to forward it to the domain that handles partitioning.

Lazy Evaluation of Flags

The optimization described in the previous subsubsection can prevent partitioning with respect to flags that will always be ignored. But we can go even further and never compute any partitioning and the value of any flag a priori. The point is to remember which instruction is the last to assign each flag. When the value of a flag is requested, the adequate instruction is replayed to deduce the useful partitioning.

The previous strategy was clearly an optimization since it has a linear cost at initialization time (which is considered to be reasonable) and is only an improvement afterward. On the other hand, evaluating lazily flags as some cost: if status flags are needed rarely enough, asymptotically, this strategy is profitable; conversely, if we use all flags, it implies an overhead. Consequently, this strategy would need to be tested before being able to know how efficient it is. Moreover, from a practical point of view, it is quite complex to implement, thus, we deemed not to try it for now.

Chapter 15

Stack Abstraction B oth C and assembly use a stack but, as it was explained in chapter 8 "Semantics of Mixed C and Assembly" (page 99), they have a very different point of view of it. This chapter proposes an abstraction of the stack that is compatible with both languages, and is expressive and precise enough for our industrial application. Globally, the stack is segmented into two kinds of blocks: C call-frames and explicit stack segments manipulated by assembly. The point is to make this abstraction as sparse as possible in order to minimize the set of additional hypothesis, and to make the abstract state as lightweight as possible.

It is good to keep in mind that we are allowed to reject behaviors that are correct according to the concrete semantics, but for which the abstraction is not good enough or too complicated. We are mainly interested into accepting behaviors that actually exist in our applications. In practice, the abstraction described in this chapter accept most legal behaviors requiring only crucial assumptions.

Abstracting Stack in a Single Assembly Block

Informal Explanation

When entering an assembly block from C, the stack is not empty: it contains the call frame of all currently active functions. We know that the local variables are contained in the stack, but we cannot tell where. For the C code after the assembly block to behave correctly, we need the stack to be the same at the end of the assembly as it was at the beginning, up to local variables that may be safely changed.

Since we do not know the content of the stack, it is difficult to guarantee that after some modifications, it is restored to its initial state. A safe criterion is not to touch the cells of the stack that were existing previously, except local variables that may be accessed by the literal x[EBP] that is an alias for the C local variable x. For that access to be legal, we add another constraint to check: EBP has not been modified before and it is restored by the end of the assembly block. We simply use a copy of EBP to remember its original value, and guarantee that it is unchanged when used or exiting the assembly 184 CHAPTER 15. STACK ABSTRACTION block.

Since we do not know what is initially in the stack when we enter the assembly block, it is pointless to read it, and we said that we want to forbid modifications in this region. Overall, we exclude any kind of access in this region of the stack: we want to ignore it completely. A natural abstraction is to represent only the suffix of the stack, initially empty and that must be empty at the end of the block.

Let us illustrate this abstraction over examples before giving a formalization. The example of Listing 15.1 pops the top of the stack and stores it in EAX, then this value is pushed on the stack. Obviously, the stack is restored to its initial state * (and only EAX is changed, which is allowed), but this code is rejected as it reads outside the suffix of the stack, especially, the POP will fail with a definite error, since it reads the unknown prefix. Such code does not exist in practice since we do not know what the POP will return, and thus, there is no point to get this value. The Listing 15.2 shows an example of a correct code that is provable by this abstraction. At the beginning of the assembly block, the stack is totally unknown, with an empty suffix. We push the integer 1 on the stack, so after the line 4, the suffix is 4 bytes long (when operating in 32-bit mode). Line 5 pops 4 bytes and stores them in EAX, The suffix is empty when we exit the assembly block, thus the stack is restored to its initial state. When we enter an assembly block, ESP is set to a + N, that is, it points after the end of the array. If we perform too many PUSH on the stack, we will entirely fill the array. At this point, the abstraction defaults to , and we shall abandon all hope to restore the original value of the stack by the end of the assembly block. We are doomed to use a statically allocated array since dynamically allocated array are much more difficult to abstract precisely. Moreover, a good choice of the size of this array prevents overflow in the abstract. Indeed, in critical embedded software, memory is a rare resource, we have to know exactly how much memory we use. This implies the choice of guidelines that promote some programming styles, and avoid others. For instance, most variables (especially arrays) are global, so that they are statically allocated at the beginning of the program, and there is no subsequent malloc that can fail. We also avoid recursive functions, as we mentioned before. Likewise, PUSH statements do not appear in loops, as it could lead to a stack overflow; in practice, they only appear outside loops (but possibly in conditional branching), and thus are executed at most once. This is the foundation of our heuristic to choose the size of the array that store stack suffixes (the constant N): the array must at least be large enough to store as many 4-byte variables as the maximum number of PUSH statements in a single assembly block † . In the examples of Listing 15.2 and Listing 15.3, we would choose a size of 4 B (1

PUSH × 4 B).
Of course, it is very easy to trick this heuristic, as shown on Listing 15.4. Here, we see only one PUSH, thus, the heuristic deduces that an array of size 4 is enough. But when we extend the stack for the second time, at line 5, the array is not long enough, thus the abstract value defaults to , for want of something better. All following statements raise an alarm since we cannot guarantee that they are error-free: includes the possibility that the suffix is empty, in which cases, lines 6 to 8 would all be faulty. Such tricky code is rarely encountered in practice, and an ad hoc fix to the previous heuristic is to add a big enough static number of cells in the array. The choice of 10 4-byte cells emerged to be satisfactory for all examined cases. It could be changed by the user if needed. The stack abstraction that we have presented is a two-stage abstraction: we abstract the stack by a stack suffix, and the stack suffix by a fixed-sized array. Let Σ be the alphabet of the stack. In the case of x86 assembly, Σ = 0, 2 8 -1 . But this method can be used for any kind of stack, though it may not be well-fitted. A stack is a word in Σ , as we exclude infinite stacks.

int f() { // concrete
For the first stage, we need an abstract domain D S that abstracts stack suffixes with γ S : D S → P (Σ ) its concretization. An abstract state s ∈ D S is an abstraction of the set of the potential contents for the blocks in the represented suffix area. This set is defined as a parameter of our abstraction by the means of a concretization function γ S . Since we keep no information about the prefix, s may also be interpreted as the set of the potential values for the entire stack (by completing a potential suffix with an arbitrary prefix). This interpretation is formalized by the following concretization function:

γ : D S → P (Σ ) s → p • s (p, s) ∈ Σ 2 , s ∈ γ S s
where a • b is the concatenation of a and b.

The second step is the abstraction of the suffix: how we build D S and γ S . ESP is abstracted, as explained in section 14.2 "Register Abstraction" (page 175), by a variable in the underlying domain. Likewise, the array is abstracted as a single array whose size is fixed and computed before the analysis. The abstract domain is simply the underlying domain, but the concretization has to be slightly tweaked, to transform the concrete array into a word such that the last cell is the first letter. This is a simple right composition by i ∈ 0, N -1 → N -1 -i to revert the order of letters.

Discussion

The preeminent qualities we expect from an abstract domain is to be precise enough to prove the desired properties on the target code and fast enough to make the analysis run in reasonable time. This abstraction meets both expectations. Moreover, it is really easy to implement since most abstract transfer functions rely directly on underlying transfer functions. For instance, PUSH and POP instructions are converted into reads and writes in the suffix array (and updates of ESP), and delegated to the underlying domain. Non-stack operations are simply forwarded to underlying domains.

The main draw back is that it does not behave well when performing a union between abstract states with suffixes of unequal sizes. In this case, the abstract value of ESP allows it to point to several cells of the array, which make difficult to prove that the suffix is empty at the end of the stack. A possible solution is to partition according to the value of ESP to keep a precise abstraction of ESP in each case.

We could relax slightly the constraints using a stronger hypothesis. If all local variables are allocated at the beginning of the function, and are all accessed relatively to EBP, ESP does not really matter during function execution. We just need to restore it correctly before executing the postlude that cleans the current call frame. This would allow more convoluted stack usage by simply saving ESP in a local variable, and restoring it at the end. Of course, while it is easy to check the stack is cleaned up, stack instructions will still suffer the lack of precision.

Evolution Possibilities

This abstraction allows only a fixed maximum length for the suffix. As explained, for our application, this is perfectly satisfactory, but we may be interested into possible evolution for other kinds of software.

The main difficulty is the representation of unbounded stack suffix, typically when PUSH statements appear in a loop (or a recursive function). In fact, there is very little difference with array abstraction, and thus, that would fall within the purview of conundrums of shape analysis. Classical abstractions that could help are modulo smashing and segment smashing.

Modulo smashing may smash all the iterations of a loop into the variables pushed by the first iteration. For instance, in the case of Listing 15.5, it is likely that push EAX always push a value that has morally the same meaning at each iteration. The same remark stands for push EBX. Consequently, mixing the value of EAX (resp. EBX) at each iteration may be an acceptable abstraction. Modulo smashing allows a segment of stack to be represented as only two 4-byte cells (one for each syntactic PUSH) and an abstract integer to keep the length of this segment of stack.

Abstracting Successions of Call Frames

Recall on Mixed Calls and Calling Conventions

Functions can either be implemented in C or in assembly and called from C or assembly: there are 4 combinations. There are two situations relatively simple, when the calling language is the same as the language of the callee. When a C statement calls a C function, the stack is no matter to be concerned about. Likewise, when an assembly function is called by an assembly statement, the stack is explicit in both sides, there is nothing special to do: assembly calls are not handled differently as jumps.

But when languages are mixed, we need supplementary assumptions to make them work together. The set of these assumptions is a calling convention (see section 6.5 "Calling Convention" (page 87)). A calling convention specifies how arguments are passed to functions, how result values are returned, whether the caller or the callee saves each register, who allocates the call-frame etc. We use a calling convention called cdecl. We recall quickly the characteristics:

- Let us illustrate on an example how we end with an alternation of C and assembly call frames thanks to the snippet of Listing 15.6. The execution starts in void f(). We are in a C function, thus, the stack contains a C call frame, but we do not care about the stack in pure C. When entering the assembly block, the stack gets an explicit suffix. In this assembly block, we call the C function int g(int), which adds a new C (thus opaque) call frame above the explicit segment of the stack. Finally, g calls assembly function int h(int), which appends a new explicit suffix atop of g's C call frame. At this point, the stack has a quite complex structure, as depicted on Then, h returns using a RET statement, which pops r g . Since the return type of h is int, EAX contains the returned value. According to cdecl, it is g's job to clean the stack from the parameters. As g is a C function, it returns using a C return statement, and since it was called from assembly, the returned value of g is in EAX. The caller is in charge of cleaning the stack.

In this example, we implicitly used a callee-saved register: EBP in f. When we use EBP to access the local variable int result, we need EBP to be restored to the value it had before call g. Since EBP is callee-saved, it is the responsibility of g to restore it. Nevertheless, EBP is modified in between, it may even been changed explicitly in h. Callee-saved registers are backed up in the call frame of g. Even if we do not know how and where they are located, we need to keep this information in the abstraction. One may remark that g may not save all registers, but only the ones that g writes. It could also save the content of such a register in a caller-saved register, like ECX. However, if g calls another function, it will be its responsibility to save ECX, just like it is its responsibility, at the end, to copy back ECX to the appropriate callee-saved register. What matters is that from the caller point of view, callee-saved registers are not restored to their original value.

Conversely, when h is called, g expects it to restore all callee-saved registers. In this case, we do not save anything since we do not change such register. But, for the execution of g to be correct, we need to check that assembly functions let callee-saved registers in the same state they have at the beginning.

The Idea

We assume we start with a C function. When we enter an assembly block, as explained previously, we represent the suffix in an array. If we need to add a new C call frame, we freeze the suffix, and we add another structure to represent callee-saved registers. These are just copies of the 5 callee-saved registers. If we call an assembly function, or we enter an assembly block, we use another array to represent the new stack suffix, and so on. We shall remark that contiguous C call frames are not separated: the only thing that matters is to restore the original state of callee-saved registers, thus subsequent C functions does not change anything.

We see that this abstraction does not make a special place for callee-saved registers in assembly functions, as there is for callee-saved registers in C functions. The moral reason is that there are many ways to save callee-saved registers, and if there are some backups, there are in the explicit segment of the stack. Some callee-saved registers may not be saved at all if there are unused, and thus, unchanged. The technical reason is that we only need to check that the value of callee-saved registers is the same when we enter the assembly function, and when we exit it, and the development version of Astrée provides a way to check that a variable has the same value at two given program points. This feature allows easy checking that calling convention requirements are fulfilled by assembly functions.

Because of technical constraints, we have to allocate statically all arrays used for explicit stack segments and all structures used to remember callee-saved registers in C call frames. This issue was already mentioned previously. We already discussed how to choose an appropriate size for explicit stack segments. Here, there is another difficulty: choosing the right number of available arrays and structure to back up registers, that is, the number of alternation between C and assembly call frames.

For that question, we found no satisfactory heuristic. We make this quantity a manually parameterizable static integer. The point is that there are complicated control flow, alternating C and assembly, but they are relatively exceptional: most of the control flow is pure C. For instance, to perform a syscall, an application program in C calls a C function that is part of system library. Ultimately, the syscall needs assembly to be triggered. From the other side, the handler of the syscall must be in assembly as well to handle precisely the content of the stack. But of course, we do not want to perform the requested operation in pure assembly, it would be painful. The control goes back to C as soon as possible. According to syscall code, the correct handler is called, and we are again in a C world. At some point, the syscall may require some assembly blocks to perform some low-level operations, but hardly more. Overall, we need two call frames of each kind, in this example.

Most execution paths require not even that many call frames. Overall, we find 3 to be a good number. For development and debugging purpose, we used only 1 call frame of each kind: fewer variables implies shorter logs, easier to read. It turns out that 1 of each kind is enough in most cases. The reason is that often, assembly functions are leaves in the call graph, or they call only other assembly functions.

Formalization

Once again, this is a multistage abstraction: we abstract a stack into a succession of C and assembly call frames, and each call frame is abstracted using an underlying domain.

Concretization is not direct though: this abstraction is not about directly abstracting the shape of the stack, but being able to restore registers correctly. But there is a myriad of possibilities to restore the registers: they can be untouched, they can be saved on the CHAPTER 15. STACK ABSTRACTION stack, they can be saved in a structure on the heap with a pointer in the stack to this structure, or to another pointer pointing to this structure. . . . For this abstraction, the concrete domain is not the simple memory state from the naive standard semantics. We need to use an enriched semantics. An enriched nonstandard semantics is an extension of the standard semantics that carries some additional pieces of information that do not exist as is. But since these pieces of information come from the standard semantics, enriched semantics are not more precise, but they may be easier to abstract precisely as they can distill relevant data that would be dissolved otherwise. The Part IV "A Reduced Product with Ghost Variables" (page 229) is about abstracting such semantics modularly with an extensive generality that we do not entirely need for now.

We can give a simple example of enriched semantics. From the standard semantics of C (or Asclepius (see chapter 4 "A Minimal C-like Language" (page 43)), or any similar language), we track the number of currently nested functions: we add a single integer to states, which is incremented on calls and decremented on returns. This integer does not carry additional information, but it makes it explicit and easier to abstract. If we are interested of the number of nested functions (for instance, for precise stack usage analysis, to prevent stack overflows), this semantics may be very useful.

In our case, the information we need to make explicit is the state of callee-saved registers along with each C call frame that follows an assembly call frame. The concrete domain is thus not only a stack, but a stack divided into segments, with half of them being annotated with 5 integers (for the 5 callee-saved registers). Formally, the concrete domain has type

D := Σ • Σ × Z 5 • Σ
The first component includes the unknown prefix and the first explicit segment. We can remark that the semantic of Panacea already took the registers to save into account: in section 8.2 "Memory Model" (page 103), the stack contains copies of the enriched heap exactly for this purpose ‡ . Thus, from the point of view of the semantics introduced in chapter 8 "Semantics of Mixed C and Assembly" (page 99), this model of stack is not enriched. This is because the concrete semantics is itself enriched compared to the actual behavior of the computer, in order to be able to detect errors like calling convention violation. In a real program, we do not have copies of register to check that they are properly managed: such a violation would not be detected, and the program might behave erratically § . Yet, even if both stack models store copies of registers, the type of stacks here and in the definition of the semantics of Panacea are not the same: in chapter 8 "Semantics of Mixed C and Assembly" (page 99) we were interested in a stack that contains values we can indeed access (explicit segments of the stack), data useful for control flow (e.g. return addresses) and the stack of local environments. In this chapter, we only want to abstract the data part. The question of return addresses is treated as a part of the problem of control flow (and is explained further in chapter 17 "Control Statements and Modification of the Iterator" (page 207)), and local environments are useless since we forbid recursive functions.

Let D S a domain that abstracts m stack segments and n integers for any (m, n) ∈ N 2 , possibly in a relational way, with concretization

γ S : D S → m∈N n∈N P (Σ m × Z n )
We define the abstract domain D the subset of D S whose concretization contains n + 1 stack segments and 5n integers, for any n ∈ N, that is

D := s ∈ D S ∃n ∈ N : s ∈ P Σ n+1 × Z 5n
and the concretization

γ : D → P (D) s →                                  p • n i=1 s i • r i • s n+1 n ∈ N, p ∈ Σ , (s i ) i∈ 1,n+1 ∈ Σ n+1 , (r i ) i∈ 1,n ∈ Σ × Z 5 n : ∃(q i ) i∈ 1,n ∈ Σ n : ∃(w i ) i∈ 1,5n ∈ Z 5n : (s i ) i∈ 1,n+1 , (w i ) i∈ 1,5n ∈ γ S s , ∀i ∈ 1, n , r i = q i , w 5(i-1)+j j∈ 1,5                                 
This is a glorified way to express that the stack is made of an unknown prefix followed by an alternation of explicit and unknown segments. The latter are annotated with 5 integers (for the values of each callee-saved register).

The underlying abstraction (D S ) is left to an appropriate domain. It must be able to abstract arrays and integer values. It is interesting to see that we do not change the type of abstract values, but we interpret them differently.

Discussion

This abstraction matches the master requirements (precision and performance). The main remaining question is whether it is future-proof: does it work on other kind of software or can it be easily improved to work?

Assembly is mixed with C code to do things that cannot be expressed in C, either because it is platform-dependent, like what OSes do, or for performance reasons. When assembly is used for performance, it seems to be rarely in the middle of C control flow, they rather are leaf-functions or just blocks in C functions. In OSes, system developers try to use assembly in these simple ways: this is the cleanest, but it is not always possible. Due to legacy code or weird architecture, there are always some weird twisted control flows, especially at startup or when we need to interact with low-level features, like exceptions or call gates. Such codes are often particular enough to be alone in their category, thus, we cannot tell if our abstraction will work without being confronted to such problem. Nevertheless, we can try to comfort ourselves with the same rationale as in subsection 15.2.2 "The Idea" (page 190): assembly is needed at some point but there are probably not that many alternations. Indeed, we have not found code that need unbounded alternation of C and assembly, so far. Moreover, such a code probably uses recursive functions, that would imply very heavy modifications in Astrée, anyway.

Another good point of this abstraction is that it relies on the underlying domain for most abstract transfer functions, thus it has all the power of the underlying domain and it is easy to implement.

Implementation

For abstracting a single assembly suffix, or alternation of call frames, we need an underlying domain to represent array and integer variables. In the hierarchy presented in section 14.4 "Implementation" (page 178), and especially Figure 14.1 "New structure of the composite domain of Astrée" (page 179), we can see that we need to be above struct domain since, as explained before, under this domain, variables are replaced by physical cells. Just like for register abstraction, we want to be as low in the hierarchy as possible, to make the domain as simple as possible. So, just like registers, stack is abstracted by the new CPU domain.

This position has another advantage: it is under parallel domain which receives from struct domain information about modified variables so as to determine the effect of each thread to others (see [START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF][START_REF] Miné | Relational thread-modular static value analysis by abstract interpretation[END_REF]). As the CPU domain forward these pieces of information, it can read it to guarantee that deep assembly call frames (explicit segments that are not the current suffix) are not modified.

Chapter 16

Other Non-control Statements S o far, we have treated most kinds of statements: arithmetic, logic and stack oper- ations. There are other kinds of instructions that are quite rare overall, but that are important nevertheless. We are still not concerned about control flow statements: this is the matter of chapter 17 "Control Statements and Modification of the Iterator" (page 207).

The most important category of remaining statements (excluding control-flow) are system instructions. They are a small group of very heterogeneous instructions. We present how we manage to implement their semantics with minimum pain.

Following sections are about other kinds of statements. Also, we will see how to handle dynamic code. This is not a distinguished type of statements, but they require a special preprocessing.

System Statements

Scope

System instructions are central in the control of memory protection features of x86 processors. These features are used to guarantee memory isolation of each process, especially, to isolate the privileged world from the user world.

Because of time constraints, we are not able to handle all kinds of system instructions. Especially, paging requires memory abstractions that are not designed yet (see section 26.1 "Paging" (page 327)). Moreover, instructions that are supported are only in the settings of our industrial application.

More precisely, we are handling LGDT (load GDT, see subsubsubsection 2.3.2.2.3 "Global Descriptor Table" (page 23)), LIDT (load IDT, see subsection 2.4.3 "Interrupts and Exceptions" (page 32)), LTR (load task register, see subsection 2.5.2 "Task Register" (page 35)), task switches (from control transfer statements, see section 2.5 "Tasks" (page 34)) and loading segment registers (see subsubsection 2.2.1.3 "Segment Registers" (page 12)). Globally, we are mainly concerned with segmentation, and especially a particular kind of segmentation: flat memory model. We want to prove that this model is correctly implemented, with right access flags, and that it is fully set up before using it.

We explicitly do not support other segmentation model as we did not need them and most systems use a flat memory model nowadays (see subsubsubsection 2.3.2.2.2 "Segmenting the Linear Address Space" (page 21)).

System Registers and CPU State

Most system instructions modify or use system registers: we had to model them. Of course, we add control registers (CR0, CR2, CR3 and CR4 as 32-bit variables) and segment registers (CS, SS, DS, ES, FS and GS as 16-bit variables). We also need some more specifics registers: GDTR, IDTR and TR. GDTR and IDTR are split into two parts: their limits (a 16-bit variable) and their base (a 32-bit variable) (see subsubsubsection 2.3.2.2.3 "Global Descriptor Table" (page 23)). They are all added into the CPU domain (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)), and represented using underlying domains.

But this is not enough. As explained in subsubsubsection 2.3.2.2.4 "Segment Selectors" (page 23) and subsection 2.5.2 "Task Register" (page 35), registers may have a hidden part to store the GDT entry they point to, so as to improve performances: GDT lies in the memory, which is slow to read; by storing appropriate pieces of information into hidden registers, the processor speeds up access to these GDT entries without allowing software to modify this cached value. Yet, we are not modeling hidden parts of segment registers; since we are limited to a single kind of segmentation model, we know what hidden parts shall contain. When a segment register is loaded, we check that the corresponding GDT entry matches the conditions and if it does, it has the expected shape, thus we do not need to copy it.

There is no such restrictions about TR, thus, we need to represent the hidden part of TR. Since a GDT entry is 8-byte long, the hidden part of TR is modeled as two 32-bit variables.

The OS that we are analyzing does not start with the processor, but it is launched by a boot loader, as well as most Linux installations use the boot loader GRUB. The boot protocol is even very close; when the boot loader launches the OS, segmentation is already enabled but paging is not. We assume a flat memory model with access rights that allow execution, otherwise, the OS would not start. Even if the model given by the boot loader is the same as the one we want to set up, we cannot just rely on initial segments. First, the boot loader is not known to provide user-level segments. Moreover, initial segments allow execution in ring 0, but we are not sure about the access rights, for instance, we do not have the guarantee that these segments do not allow execution of system code from ring 3; this is not a problem as long as there is no user-world task, but it should be fixed before creating them. Finally, GDT does not only store segments, but also other kinds of system descriptors that have to be set by the OS, like the IDT (see subsection 2.4.3 "Interrupts and Exceptions" (page 32)). Since we do not know a priori the shape of the GDT the boot loader built, we shall rewrite everything.

In our initialization scenario, we need to set the GDT (using a LGDT instruction)

SYSTEM STATEMENTS

197 before loading segment registers. Since the original GDT is unknown, loading a segment register before setting up the new GDT would give an unknown result, and possibly an error (if outside GDT limit). By doing so, we reject concrete traces that are legal but that cannot be abstracted meaningfully, since each subsequent memory access could do anything. These unwanted traces do not exist in the analyzed OS, and are not sensible.

To remember whether the new GDT has been set, the new CPU domain stores a Boolean value that tells whether a LGDT statement has been executed. This is an important, synchronizing event. We do not allow binary operations (like join or meet) between states with unequal values for this Boolean variable. Once again, this restriction rejects legal traces that cannot appear in any reasonable code.

Microcode

Until now, we only talked about system data structures and registers, but not the semantics of system instructions. Their behavior is described with pseudo code in [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF], their components are quite simple, but the complexity comes from the length of this pseudo code. Even if the expressions and elementary statements are simple, they are very long to write in the internal format of statements in Astrée: we need to specify explicitly the full type at each node of expressions, fill some fields that are not all relevant. . . . Moreover, in the abstract, testing a condition is much more complicated: we need to guard by the negation of the condition and test if the result is ⊥. Translating the pseudo code of simple instructions (∼ 20 lines of pseudo code) results in more than 150 very long lines (∼ 300 columns) in OCaml code. Writing complex system instruction whose pseudo-code is more than 4-page long seems unfeasible without a great risk of errors.

In practice, statements in system instruction pseudo-codes are a very particular subset of internal statements. From this remark, we defined a language, as OCaml datatype, that is merely a shorthand for writing internal statements.

Let us describe a simplified version of this language, starting by the structure of expressions and lvalues. We ignore aspects that are only shortcuts or debugging features, though the weary developer will not miss appreciating them. Leaves in expression trees are either Name(_) or Expr(_). The latter is used to introduce an expression of the internal format of Astrée. We use it mainly to introduce variables. The former type of leaves contains a name that is an alias for another expr : this language is executed with an environment that stores map of aliases to share complex expressions. These maps also contain the operands of assembly instructions, usually aliased by names dest and src. Deref(e, o, s, u) is used to dereference a number of bytes specified by s (BYTE = 1, WORD = 2 and DWORD = 4), at address e + o (e is the base pointer and o an integer offset in bytes), interpreted as an integer with signedness given by u. Bits(e, f, n) is used to select n bits from e, starting at bit number f. Other nodes are quite clear. The only surprising point in the syntax of conditions is that conjunctions and disjunctions are not limited to 2 operands. We can also note there is no "Not" node, since it is a source of mistakes: a correct approximation of "Not(a)" shows whether a may be false, but we are often more interested in the case where a is definitely false * . Static conditions are conditions whose leaves are names, that are aliases in the environment for Boolean values. They are very useful to parameterize the semantics of instructions: when two instructions have very close semantics, they can share the same code with a static condition to branch between two variants, thanks to a Boolean value that is set or cleared depending on the instruction we want to run. For instance, codes for task switches have a lot in common, but differ depending on whether the task switch comes from a call, a jump or a return. The microcode is common to all these flavors, but it contains static conditions to enable statements only when appropriate. Here, we allow a "not" node since these Boolean values are not part of an abstract state.

exception ::= Custom | intel_exception
We define two kinds of exceptions: standard exceptions planned by the semantics, and a custom exception for additional checks that are relevant only for our analysis. Fail(e) raises exception e. Assert(c, e) evaluates condition c and raises the exception e if it can be false. It can be rewritten as If(c, Skip, Fail(e)), where Skip is a statement that does nothing, like Sequence(). Call(f) runs the statement aliased by f in the current environment and then comes back to the next statement. AddAlias(name, e) adds name as an alias for e in the current environment. Switch and StaticSwitch take a list of pair of conditions and statement, with an additional statement for the default case. The executed statement is the first whose condition is true. Ignore is used to replace the current abstract state by ⊥. It is useful as the default case of an exhaustive Switch † . Absurd asserts that the current abstract state should be ⊥, otherwise, there is an error in the microcode ‡ . It is useful as default case of a StaticSwitch.

Overall, in addition to the abstract state that we are processing, a memory state in the microcode is made of 3 maps: names to expressions, names to lvalues, and names to (static) Boolean. The microcode of an instruction is a statement with a map that associates a statement to each f un_name . Thanks to this language, we can write formally the semantics of instruction is a form very close to the pseudo-code we find in Intel's manuals. Unfortunately, since the pseudo-code is not very formal (in particular, it contains many natural language descriptions), we cannot automatize the translation from pseudo-code to our microcode.

Dynamic Code

Dynamic code is not a recommended programming style, but we found some in real world program. Sometimes because it is a light fix avoiding a huge restructuring, or because technical constraints make it difficult (or inefficient) to do otherwise. It would be foolish to believe that it is an old habit that does not exist anymore. Dynamic code is introduced because it was necessary, not because it was fashionable. We should not say that modern OSes do not have dynamic code, but that they do not have dynamic code yet.

Since it exists in real-life code, we shall support it in our analyses, at least partially.

A Simple Strategy

Typically, dynamic code is written into an array, then a jump statement transfer the control at the beginning of the array. A first approach is to concretize the content of the array, disassemble the result to get the instructions. For efficiency concerns, we should take some precautions in this process: concretization may return a very large set. We want to detect this early enough to raise an alarm. Even if the concretization of each byte gives a reasonably small set, Cartesian product may still be huge. If both these operations succeed by giving a reasonably small set of possibilities, we can forward the result to a disassembler. Disassembling is performed by Amical (see chapter 9 "Amical" (page 121)). Each sequence of byte is translated into an instruction and a sequence of remaining bytes, by iterating, we get all the instructions to execute. Possible sequences of instructions should be run in parallel, and abstract states joined at the end. We shall also remark that such dynamic code should contain a jump statement otherwise, we would reach the end of the array, at which point, fetching the next instruction is an illegal memory access, at least, in our restricted semantics.

This strategy seemed to be sufficient for analyzed sections of the code. But it has a lot of limitations, especially about the size of concretization.

Improving Amical

Some instructions may be very simple, but leading to concretizations with very poor precision. For instance, we dynamically write byte E9H in an array, at address a. E9H is the opcode for a relative near jump with a 32-bit offset, which is interpreted relatively to the next instruction, i.e. the end of the current instruction. The opcode shall be followed by 4 bytes. The end of the instruction will be at address a + 5. This instruction will transfer the control to address a + 5 + o where o is the specified offset. To jump to a given label l, we need that a + 5 + o = l, i.e. o = l -a -5. Let us write o on 32 bits after the opcode. We get a 5-byte statement that jumps unconditionally to l. An example of such code is shown on Listing 16.1.

Address of labels are often unknown, and concretizing a byte sequence such as the example would give a perfectly unknown offset, while a symbolic domain can have a very precise abstract value. The reason is that the real byte sequence does not matter; we only need to know a more symbolic property on the value defined by the byte sequence.

To handle correctly such cases, we need a less eager disassembling. We disassemble just enough bytes to get the prefixes and the opcode. With this information, we can directly use parts of the array as immediate operands or displacements, without having to concretize them. For memory and register operands, we still need to concretize the Mod-R/M and SIB bytes. In Listing 16.1, we would start by concretizing and disassembling one byte: we get a precise value, from which we can tell that the instruction is a simple relative jump, whose offset is contained in the 4 following bytes: *((int*)(a + 1)). A domain such as Astrée's struct is able to remember that these 4 bytes make a single cell whose content can be represented using any underlying domain (like linear combinations domain). This solution is not implemented yet but it will probably be as it is required to analyze code such as the snippet of Listing 16.1, that exists in not-yet-analyzed parts of our industrial application.

There is another point on which Amical can be improved. When we need the instructions dynamically written in an array, we said that we can concretize each cell to get all possible sequences, which may lead to a huge set of sequence (the Cartesian product of each cell). A solution is to make Amical to accept sequences of set of potential bytes (rather than a single precise sequence): by consuming each cell consecutively, Amical can reject huge sets of illegal sequences. For instance, if a prefix appears twice, the instruction is not valid, whatever the following. There is a limitation: this method does not allow relations between bytes. A satisfying approach may be to apply this method with a coarser granularity, by providing independent blocks of bytes § . § Typically, an offset and an immediate operand are likely not to be related, and they can be seen as two independent 4-byte block. Of course, there are relations between the bytes of each of these values.

Floating Point Instructions

Early processors of the x86 family can use a coprocessor for floating-point operations. The first was designed to be used with the 8086 and was called 8087. Consequently, this family of coprocessors took the name of x87. Following x86 processors had their x87 coprocessors. The 80387, math coprocessor of the 80386, was the first fully compliant with the IEEE 754 standard. The 80487 was already like modern processor: it contains a complete processor and, when installed, it deactivates the main 80486 to run all instructions by itself. Nevertheless, it still needed the 80486 to be present. Math coprocessors did not last very longer as the 80587 was the last x87. Since then, the floating-point unit (FPU) is always integrated in the main CPU.

FPU instructions are mostly straightforward, and not very different from C's math library. The main difficulty in the FPU is the memory model: it uses a cyclic stack of 8 registers. Registers are not named absolutely, but relatively to the current top. This very particular structure is not as straightforward to abstract as other registers.

Since FPU instructions are not very relevant in the context of an OS, this problem has been decided to have low priority and is not treated yet. With the next section and paging, they form the main categories of statements that are not handled and would probably require heavy work.

Input/Output

In addition to access to the RAM by the memory bus, processors of x86 family can transfer data through input/output (I/O) ports. Ports are created by the hardware circuitry from processor pins and then configured to access external devices. By I/O ports, the processor can control these external pieces of hardware, receive data from, or send data to external storage. . . . I/O ports are controlled by their own mechanisms of privilege levels that we will not explain in detail. Receiving and sending data is done respectively by instructions IN and OUT, and their string instruction counterparts INS and OUTS. These instructions have two operands: the port number (always stored in general-purpose register DX), and the register (always AL/AX/EAX for simple versions) or memory (always [ES:(E)DI] for string versions).

Modeling correctly these instructions requires knowing hardware circuitry around the processor and specifications of devices with which we are communicating.

In our application, we do not have all these hypotheses. But, in analyzed parts of the target OS, I/O instructions are used in a very restricted way: to set up an external timer to trigger the scheduler (see section 2.7 "Boot" (page 37)). For want of a precise specification of this timer, we could not make a good abstraction. Instead, we choose to compare I/O port accesses with an expected scenario.

A communication scenario is a (usually finite) sequence of tuples made of a value of a two-valued type (whether it is an input or an output), a port number and an integer. This is scenario is an a priori knowledge. The semantics of I/O instructions with respect 16.4. INPUT/OUTPUT 203 to a scenario are quite direct. For IN, we check that the next step in the scenario is indeed an input on the appropriate port, the result is set to the integer value planned by the scenario, and we pop this step from the scenario. For OUT, we check that the scenario expects an output on the given port, then we check that the expected output is indeed the value of the operand. Likewise, we remove the executed step. For our application, at the end of initialization, we expect that the remaining scenario is empty. For instance, Listing 16.2 shows a code that reads 4 bytes from I/O port 42, adds 1, and outputs the result to port 43. This code matches any communication scenario that expects an input from port 42 of a value n and an output to port 43 of a value n + 1, just like the scenario shown in Table 16.1. Indeed, when we run the first line of the code, we check that we indeed read a value from port 42, and EAX is assigned the value 5. We remove the first line of the scenario. The second line of the code has no effect on the communication scenario. The third line is an I/O instruction: we check that it matches the requirement of the first line of remaining scenario. It should be an output to port 43 of the value 6, which is correct. At the end of the code, the remaining scenario is empty, thus the initial scenario is satisfied. It is easy to design an abstract domain to check a communication scenario: it simply holds an integer that is the progress in the scenario. We reject the union of two states with unequal progress: like setting the GDT, these communications are important synchronizing events. If such a union occurs, the abstract state becomes and is removed with an alarm. During the abstract execution of an IN or OUT instruction, if a check does not hold, we raise an alarm and remove these traces. If such an instruction is executed when the communication scenario reached the end, we also raise an alarm.

Such a basic check is useful to verify that all execution traces go indeed through the same communication process. In addition, specifying the communication process can be useful to check that is match the specification of the external device, either manually, or by another analysis, centered on the device. Such a modular analysis requires to write the communication scenario by hand.

A very simplistic scenario has been hard coded in the CPU domain to study the feasibility of this method. Results were encouraging, and we trust this method can indeed be used for the initialization sequence of our application. This proof of concept was then removed as it is in fact useless, and tried to check the toy scenario on every program. A proper implementation requires to allow the scenario to be specified as external configuration or annotations in the source code. Due to time constraints, this is not implemented for now: making the scenario parameterizable is a much bigger work than implementing the transfer functions for I/O instructions.

To check the initialization sequence of our study case, we will need to support I/O instructions, but we need to know the expected scenario. Since it depends on external hardware, which was not communicated precisely enough by our industrial partner, we cannot design the scenario. We may still craft it from what the study case does, but building the specification from the implementation we want to check has very little interest. It can only check that communications are indeed synchronizing events (communications cannot be skipped or reordered). Considering the estimated amount of work for the implementation, we deemed that this weak property was not interesting enough to spend more time on it for now.

We shall still note that the communication scenario does not model entirely the hardware. Indeed, it does not take timing into account: each time a line of the scenario is consumed, the next is ready. This is not actually true: external devices may need some time to provide the next value. Some communications are indeed timing based (and thus, we need a model with a notion of clock), but some are more asynchronous: an interrupt is triggered when there is something to read. For this kind of communication, we just need to remember that data are available as soon as the interrupt is triggered. This communication method seems safer, but unfortunately, this is not what our study case does during initialization. Indeed, such interrupt allows suspending most computations, which is dangerous during system set up; for that reason, these interrupts are often disabled during initialization.

String Instructions

Some instructions can have a prefix to repeat them automatically; they have a mnemonic that ends with a S (MOVS, CMPS, SCAS, LODS, STOS, INS and OUTS). They are called string instructions since they treat contiguous blocks of data. They all use DS:ESI as pointer to the source operand and ES:EDI as pointer to the destination operand (if applicable).

There are 3 repetition prefixes: REP, REPZ (and the alias REPE) and REPNZ (and the alias REPNE). REP repeats the string instruction as long as ECX = 0, and ECX is decremented at each iteration. REPZ (repeat if zero) (or REPE: repeat if equal) also decrements ECX, but the condition is ECX = 0∧ZF = 1. This condition allows repeating the operation at most ECX times, but stopping as soon as the ZF flag is cleared, that is, the result of the string operation is not zero (or if both operands are not equal, depending on the instruction). REPZ (repeat if non-zero) (or REPE: repeat if not equal) does the same, but the condition is ECX = 0 ∧ ZF = 0: it repeats the given instruction as long as the result is not zero (or operands are not equal).

The main benefit, other than readability, of using a string instruction rather than

STRING INSTRUCTIONS

205 a classical loop is performance. Otherwise, especially from a semantic point of view, it is equivalent to classical loop statement. Translating string instructions into a forstyle loop whose body is an elementary instruction is probably good enough, both in performance and precision. For now, string instructions are not implemented yet in Astrée, due to time constraints and, as they are not present in analyzed sections of the target OS, they have not been judged to have high priority.

Chapter 17

Control Statements and Modification of the Iterator

U ntil now, we were interested in most categories of statements expect control trans- fer statements. Yet, without them, control flow is totally linear and programs are mostly uninteresting. Such statements allow programming languages to be Turingcomplete, as they hold the power to select and repeat operations. The main drawback is that they often make analyses difficult.

Assembly control transfer statements may not seem particularly difficult to handle. The complexity comes from the interaction with C, since they have a very different point of view on control flow. In C, control flow structures exist even in the syntax. The only kind of transfer that is not well parenthesized are goto-statements. Yet, their destination is static (in standard C) and they only can reach labels in the current function. Contrariwise, assembly has no syntactic construct to reflect control flow, and control flow is subjected to very few rules, in particular, destination of control transfer statements are not always statically known.

As most control flow is done in C code, we need to keep all the precision we have. An even better property is to make it so that pure C code analysis is unchanged. While assembly control flow is rare, it has a significant importance and it is not limited to simple cases. On the contrary, regular control flow is rather implemented in C as much as possible. The remaining cases are mostly difficult ones.

Computing the Destination

We already mentioned the problem of the translation between code pointers and program points. A code pointer is an address in memory that designates a point into the code (and not a variable). The processor works with code pointers, since code, like data, is just a sequence of bytes, without structure. There is no a priori separation between 208CHAPTER 17. CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR consecutive statements * . Program points are logical points in a program, especially visible from the source code. They are positions between two statements. This is what semantics and analyzers work with. A destination is designed by a code pointer, but to continue the analysis, we need to translate it into program points. In x86, from the binary encoding and without context, we cannot tell where program points are. We need at least to know where the first instruction starts. Worse, not respecting these program points is allowed: the processor has no knowledge about the boundary of instructions.

Reading instruction with such a shift is a hacker technique, or used in very advanced compression. In mainstream code, it is not a desirable behavior. When we have the correspondence between the binary sequence and the assembly source code, usually, we do not allow jumps whose destination is not between two statement. This is the case here: even if the processor would not complain, we forbid such jumps; when they happen, an alarm should be emitted.

Yet, this translation is not trivial. In section 5.5 "A More Symbolic Semantics" (page 77), we needed the start address of the code in memory, and we used an oracle giving the length of each instruction to be able to resolve code pointers into program points. This is the first difficulty: we do not know precisely the length of each assembly instruction. The second imprecision source is more intrinsic to abstract interpretation: a code pointer can be imprecise.

Possible sizes of assembly statements are computed statically using Amical (see chapter 9 "Amical" (page 121)). From these individual sizes we compute a function that, given a program point and an offset relative to the program point, computes possible program points corresponding to the code pointer, and whether it can be an illegal pointer. A code pointer is illegal if it is before the beginning of the block, after the end, or if it may be in the middle of an instruction. Just like data in C, assembly blocks are supposed to live in separate universes: as we cannot craft a pointer to a variable from the address of another variable, we cannot designate a program point in an assembly block from a program point in another block and an integer offset. For obvious performance reasons, this function memoizes even intermediate computations. Formally, let b ∈ N be the number of assembly blocks in the program and for all i ∈ 1, b , let l i ∈ N be the number of assembly statements in the i th block. For all i ∈ 1, b and j ∈ 1, l i , let S i,j ⊆ N * be the set of possible lengths of the j th instruction of the i th block. The sets S i,j are computed by Amical. A program point is identified by a pair (i, j) ∈ N 2 where i is the number of the block and j the number of instructions before the program point. The first program point of a block i is thus (i, 0) and the last is (i, l i ). We define the function pp :

N * × N × Z → P (N * × N) (i, j, o) →                    (i, j ) ∈ N * × N ∃(s k ) k∈ j+1,j ∈ N j -j : o 0, j j, ∀k ∈ j + 1, j , s k ∈ S i,k , j k=j+1 s k = o                    ∪                    (i, j ) ∈ N * × N ∃(s k ) k∈ j +1,j ∈ N j-j : o 0, j j, ∀k ∈ j + 1, j , s k ∈ S i,k , j k=j +1 s k = -o                    pp(i, j, o)
is the set of program points that can be at byte offset o of program point (i, j). If we had access to the assembled version of assembly blocks, we would precisely know the length of each instruction, that is, for all i ∈ 1, b and j ∈ 1, l i , S i,j would be a singleton. In this case, we would be able to know exactly the program point corresponding to a (precise) code pointer. More formally, images of pp are only singletons or the empty set.

210CHAPTER 17. CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR

We also need the predicate invd :

N * × N × Z →{ff , tt} (i, j, o) →        o 0 ∧∃j ∈ N : ∃(s k ) k∈ j+1,j ∈ N j -j :        ∀k ∈ j + 1, j , s k ∈ S i,k ∧ j -1 k=j+1 s k < o < j k=j+1 s k               o 0 ∧∃(s k ) k∈ j+1,l i ∈ N l i -j :        ∀k ∈ j + 1, l i , s k ∈ S i,k ∧ l i k=j+1 s k < o               o 0 ∧∃j ∈ N : ∃(s k ) k∈ j +1,j ∈ N j-j :        ∀k ∈ j + 1, j , s k ∈ S i,k ∧ j k=j +2 s k < -o < j k=j +1 s k               o 0 ∧∃(s k ) k∈ 1,j ∈ N j :        ∀k ∈ 1, j , s k ∈ S i,k ∧ j k=0 s k < -o       
invd(i, j, o) is true if and only if the code pointer can be outside the assembly block, or in the middle of an instruction. The first and third cases correspond to offsets that may be in the middle of an instruction. Second and fourth cases test whether the offset may be out of the block. We are interested into resolving program points expressed as the sum of another program point and an offset because we cannot build a code pointer ex nihilo, as well as pointers in C (to variables) or functions must be based on the "address of" (&) operator. The assembly equivalent is to use a label: they are the only way to get a valid code pointer. Moreover, each label corresponds to a single program point.

Because of this similitude, code pointers are abstracted in the same way as C pointer, as described in [START_REF] Miné | Field-sensitive value analysis of embedded C programs with union types and pointer arithmetics[END_REF][START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF]. C pointers are made of a base (a variable, an array, a structure) and an offset inside the block it points to. The abstract counterpart of such a pointer is a set of bases and an abstraction of a set of integers. The same holds for code pointers. Bases are reference points: they can be any assembly program point. In practice, we cannot directly address any program point: it has to be labeled, or pushed on the stack by a call instruction. We can illustrate such abstraction with Listing 17.1. EBX is used as a counter, and EAX stores a code pointer. After line 2, EAX has a value relative to label with offset 0. After the line 3, we added an offset of 3, then EAX is the pointer label + 3. In the abstract, it is represented by a set of program points containing label and an abstract set of integers such that its concretization includes 3 (and only 3, if the abstraction is precise). Thanks to pp, we can translate label + 3 (and other code pointers in the concretization of EAX) to program point so that we can translate this dynamic destination into a set of possible destinations. Thanks to invd, we can check whether label + 3 might not designate a program point.

Like C function pointers that must be concretized to get pointed functions, we need to concretize code pointers. Function pointers are quite safe to concretize: there are no more elements in the concretization than the number of function in the code. For code pointer are made from a label (that is easy to concretize), and an offset. These labels are no more than the number of program points in assembly blocks, which is small enough not to be worried about. But, we should take care, when concretizing the offset part, to avoid too large (or infinite) concretizations. But we can bound the interval of interesting offsets from statement (i, j): the minimal offset is

- j k=0 max S i,k
and the maximal is

ln k=j+1 max S i,k
Offsets out of these bounds are out of the assembly block and necessarily invalid. Proper implementation should only consider offsets within these extrema to avoid catastrophic concretization.

In Astrée, since code pointers are very similar to C pointers, they are implemented in the same way, and especially in the same domain: they are both handled by the pointer domain (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)). The main modification is to add assembly program points to the set of 212CHAPTER 17. CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR allowed bases and to adapt handling of other kind of pointers: concretizing an abstract pointer as function pointer may be invalid when it contains assembly bases; dereferencing a data pointer may also be invalid if there are assembly bases. This is just the first step to execute control transfer statements. Once we have the possible destination program points, we still need to actually perform the jump. Depending on the destination, the jump is not handled in the same way. We distinguish several kinds of jumps. Among near jumps (see subsection 2.4.1 "Near Calls" (page 29)) we make two cases, depending on whether the destination is in the same C function as the jump statement. Far jumps (see subsection 2.4.2 "Far Calls" (page 30)) are preceded by other assignments and checks.

Since destinations are imprecise, a single statement can have destinations in the same function and in others functions. This is not a problem: the jump statement for each destination is performed separately.

Intra-function Near Jumps

The simplest case is when the destination is in the same C function as the jump statement. Once we know the destination program point, this is not very different from a C goto-statement. They are thus handled in the same way. We will here explain how it works for C gotos, as it is slightly simpler, before explaining the differences to take into account.

The Case of Gotos

The method is to store the current property aside when we execute a goto, and to restore it when we reach the corresponding label. We also need an additional accumulator to save the context on return-statements. For the sake of the formalization, we assume that all return-statements of a function return the same variable. This can be ensured using a simple rewriting.

Formally, let D S be the concrete domain of states (typically, sets of pairs made of a stack of environments and a heap). We define

D := D S × (L → D S ) × (L → D S ) × D S
where L is the set of labels. This is the domain that we use to resolve gotos. The first component is the current state (also called direct flow). The second component is the map of states to restore for each label. The third component is where we store the current state while encountering a goto. The last component is the union of states where a return happens.

When we run a goto-statement, we save the current state. The current state become ⊥ S as the program point following a goto is unreachable.

goto l; : D → D (d, m, g, r) → (⊥ S , m, g [l → d ∪ S g(l)] , r)

INTRA-FUNCTION NEAR JUMPS

213

When we reached the corresponding label, the provided property for this label is restored into the current one.

l: : D → D (d, m, g, r) → (d ∪ S m(l), m, g, r)
And finally, to run a return-statement we simply save the current flow, while the current flow is killed.

return x; : D → D (d, m, g, r) → (⊥ S , m, g, r ∪ S d)
Other unary operations, like assignments and guards, are performed only on the current property. Any function f S : D S → D S can be lifted to D → D: This allows reaching labels in otherwise unreachable blocks, like in Listing 17.2. Initially, the state is totally empty:

f : D → D (d, m, g, r) → (f S (d), m, g, r) 1 int f() { 2 int i = 0; 3 goto l; 4 if(0) {
(∅ S , l:⊥ S , l:⊥ S , ⊥ S )
where ∅ S is the state with not living variable. This is not ⊥ S since it does not mean that the program point is unreachable. After line 2, the state is

((i → 0), l:⊥ S , l:⊥ S , ⊥ S )
At line 3, we save the current property in the appropriate map.

(⊥ S , l:⊥ S , l:(i → 0), ⊥ S )
We can see that the program point after the goto-statement is unreachable, since the direct flow is ⊥ S . The condition of line 4 requires applying a guard. This means that it 214CHAPTER 17. CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR is performed only on the current property. Thus, the state after line 4 is unchanged. At line 5, we restore the saved property for the label l:. But the map of states to restore (the second component) is empty. Thus, we keep (⊥ S , l:⊥ S , l:(i → 0), ⊥ S ) Likewise, at line 6, we increment i in a bottom state: it stays the same. At line 7, we have to compute the union of the state that did not enter the if and the state at the end of the body of the if, that is:

(⊥ S , l:⊥ S , l:(i → 0), ⊥ S ) ∪ (⊥ S , l:⊥ S , l:(i → 0), ⊥ S )
Binary operations are performed pointwise, the property after line 7 is:

(⊥ S , l:⊥ S , l:(i → 0), ⊥ S )
Then, after the return (⊥ S , l:⊥ S , l:

(i → 0), ⊥ S )
We can see that the first iteration returns nothing. This is because nothing was restored at the label l:. Indeed, the map of states to restore is initially empty. But we can see that the map of saved states is not empty. This means we need to perform a new iteration of the body of the function with newly collected states. But this time the initial property is

(⊥ S , l:(i → 0), l:⊥ S , ⊥ S )
The second iteration looks like the first one until line 5: this time, there is a property to restore. After line 5, we get

((i → 0), l:(i → 0), l:(i → 0), ⊥ S )
This time, the incrementation has an effect; after line 6, we have:

((i → 1), l:(i → 0), l:(i → 0), ⊥ S )
After line 7, we have to compute:

(⊥ S , l:(i → 0), l:(i → 0), ⊥ S ) ∪ ((i → 1), l:(i → 0), l:(i → 0), ⊥ S )
which is

((i → 1), l:(i → 0), l:(i → 0), ⊥ S )
This time, there is a direct flow when we reach the return statement, thus, at the end of the function, we get

(⊥ S , l:(i → 0), l:(i → 0), (i → 1))
Since we are always returning the variable i, we know which value to extract in the returned property. This second iteration is stable since we can observe that the initially

1 int f() { 2 int i =
m:⊥ S , ⊥ S
Since the map of saved jumps is not equal to the provided map of states to restore, we need to iterate further. In particular, we can see there is a jump to label l that was not known before.

For the second iteration, we start with this state

∅ S , l:(i → 1) m:⊥ S , l:⊥ S m:⊥ S , ⊥ S
At line 6, we restore the state for label l. We get the state

(i → 1), l:(i → 1) m:⊥ S , l:(i → 1) m:⊥ S , ⊥ S
Then, i is incremented, and, at the end of the function, we get

⊥ S , l:(i → 1) m:⊥ S , l:(i → 1) m:(i → 2) , ⊥ S 216CHAPTER 17.

CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR

Once again, there new possible jumps. In particular, there a jump to label m that was unknown before. We iterate the analysis again with the initial state

∅ S , l:(i → 1) m:(i → 2) , l:⊥ S m:⊥ S , ⊥ S
At this iteration, we have something to restore at label m. Thus, after line 4, we have the state

(i → 2), l:(i → 1) m:(i → 2) , l:(i → 1) m:⊥ S , ⊥ S
This time, the direct flow is not empty at line 5: we have something to return.

⊥ S , l:(i → 1) m:(i → 2) , l:(i → 1) m:⊥ S , (i → 2)
We finish this iteration, the state for label l is restored, i is incremented, and the state is saved for label m. We obtain:

⊥ S , l:(i → 1) m:(i → 2) , l:(i → 1) m:(i → 2) , (i → 2)
This time, we have reached a fixpoint since the map of saved states is the same as the provided map of jumps. We can also see that the end of the function is definitely unreachable and that the function returns the integer 2.

Formally, the semantics of one iteration of the body of a function is a map

f : D → D
Let d ∈ D S the property at the top of the function. This is ∅ S in the examples, but it can be non-empty if the function has parameters or if there are global variables. We are interested in the least fixpoint of

f : D L S × D S → D L S × D S (g, r) → (π 3 (f (d, g, g, r)) , π 4 (f (d, g, g, r)))
The least fixpoint of f allow to get the actual returned value, where all gotos have been stabilized.

We can see that this semantics looks different to what we formalized in Part I "Semantics of Mixed C and x86 Assembly" (page 41). In Panacea, when we encounter a jump, we simply update the current program point. This is a very intuitive interpretation of gotos, and it works well in an operational semantics. To formalize an abstraction, it is easier to start from a denotational semantics, which is closer to the formalism of this section.

The equivalence between both formalisms for gotos is quite direct: rather than updating the program point to the destination (that is forwarding the current state directly

INTER-FUNCTION NEAR JUMPS

217 elsewhere), we save the current property, and we enumerate program points cyclically until we reach the destination, where we restore the property.

It is easy to get an abstraction from this formalism: we need an abstract domain D S , that abstracts D S , and we change the transfer functions and union to their abstract counterparts, and we replace the least fixpoint by a correct approximation of it.

For implementation purpose, we may propose a few improvements. The major one is to use a single map to save and restore states, without waiting for the next iteration. With this optimization, the domain has type

D := D S × (L → D S ) × D S
In addition to the memory gain, this optimization allows forward jumps to be resolved in a single iteration. In other words, we perform computations as soon as possible, rather than systematically waiting for the next iteration. Soundness of this improved method can be proved by seeing it as an instance of chaotic iterations (see [START_REF] Cousot | Méthodes itératives de construction et d'approximation de points fixes d'opérateurs monotones sur un treillis, analyse sémantique des programmes (Iterative methods for construction and approximation of fixpoints of monotone operators on lattices, program static analysis[END_REF]). The point of chaotic iterations is to arbitrarily reorder computations in the resolution of a system of equations, but to do them anyway. Eventually, we reach the same solution.

Handling Assembly Jumps

We use a very similar technique to handle assembly local jumps, but with few differences.

On the jump side, the destination may be imprecise. The current property must be saved for each of possible destination. Using pointers (explicitly or not) in the data structure that saves properties allows sharing the same structure for each possible destination, allowing efficient storage.

In assembly, destinations are not only labels: they may be arbitrary program point. We must check at each computation step whether there is a program point for the current program point, and restore it if applicable. Assembly labels are only a special case. Otherwise, everything is the same, especially stabilization of non-forward jumps † , except that we need to stabilize both maps of properties saved by C gotos and by assembly jumps.

From an implementation point of view, this is implemented in the environment domain (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)). The underlying domain (i.e. trace domain and below) forms the domain D S that abstract a single state. For practical reasons, the maps of saved states for C gotos and assembly jumps are disjoint. This is a precaution not to accidentally change the behavior of the analysis of pure C programs. Moreover, it is easier to make OCaml's typer happy this way, as the domain (the type of keys) of both maps are different.

Inter-function Near Jumps

We have treated the case of jumps whose destination is in the same function as the jump statement. But they can also be in another function. There are implementation † Either backward or horizontal jumps.

218CHAPTER 17. CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR

constraints: Astrée does not support recursive functions, thus we cannot treat all jumps like a C call. Moreover, flavors of jumps (JMP, CALL and RET) are not reliable hints on the control flow. A simple jump can be done with a RET, as well as what are morally a call to and a return from a procedure can be done using JMP only. In the following, we will only talk about undiscerned jumps, since they can all be reduced to JMP with some stack operations; the control flow part stays the same. Another difficulty is that, sometimes, there is not such strict interpretation of calls and returns: an assembly function f can jump to a function g, then g jumps to h and h returns to f. The first jump can be understood as a call whose return would be the last jump, but the jump from g to h has no clear meaning. Such cases are not only feasible, they appear in real-world code. Indeed, more regular control flows are done in C, as much as possible. Overall, it is useless to try to impose a C conception of control flow to assembly code. This problem has been explored in chapter 7 "Pathological and Didactic Examples" (page 91). We also remember that we should preserve pure C analyses.

The method used for gotos and local jumps allows mixing well-structured C with poorly-structured jumps. Unfortunately, this is limited to a single function. The idea is to adapt this method to inter-function jumps. The first step is to recall how we detect recursive calls.

As Astrée uses a single variable for each local variable, we should not have two instances of the same syntactical local variable. That's why Astrée does not support recursive functions. This is not a problem since they are not allowed in embedded critical software as they easily lead to a memory usage difficult to predict: recursive calls are like dynamic allocations on the stack. Astrée cannot only assume that analyzed programs do not have recursive calls: it has to warn when they happen, after which Astrée can kill such traces ‡ . To detect recursive calls, we need to have some kind of stack abstraction that keeps, at least, functions in the current call stack. Partitioning domain (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)) is in charge to build a trace abstraction from a state abstraction. It abstracts traces in a very light way: they are a list of events on the trace (like active calls § , whether conditions of if-statement are successful, the number of unrolled iterations in loops. . .) with the last state of the trace. Older states are totally forgotten. When a call is performed, the iterator asks the composite underlying domain to filter traces that already have an active call to the same function in their history. If there are some, an alarm is raised, and these traces are ignored: the analysis continue with traces without recursive calls. This is the criterion according to which we classify jumps: whether the destination is in the call stack or not. If it is not, we should add it; these are "forward distant jumps" ¶ . If the destination function is already in the call stack, adding it would be like a recursive function, thus we need to wait to return until we reach this destination function again ‡ More generally, we can reject any unwanted behavior, as long as we detect when they happen, and warn the user about it. The only constraint is that the analysis should still be useful for our applications.

§ That is, calls have not yet returned. ¶ This terminology is not standard. It was chosen specifically these notions that have little meaning outside the context of this work.

INTER-FUNCTION NEAR JUMPS

219

to handle it; these are "return distant jumps". We can easily classify whether a distant jump is forward or return using the partitioning domain; this classification must be done dynamically * * .

A forward distant jump is not that different from a regular C call, but it does not always include information needed for return (if it is performed by a JMP or RET) and the state at call-site is not injected at the beginning of the function (it is ⊥ here), but on the destination of the call. This is very similar to a local jump: the current state becomes ⊥, but we saved it to be able to restore it when appropriate. But, in the case of forward distant jumps, the saved state does not come from the previous iterations of the function, but on the analysis of another function. Before being able to give an example, we shall see how to handle return distant jumps.

Return distant jumps are dual of forward distant jumps, and they differ in the same way from regular C returns, as forward distant jumps differ from calls: return distant jumps may not use return information, and the property at jump-site is not returned at a hypothetical call-site (which may not exist). The current property is saved to be restored at the suitable destination, but this backup has no use in the current function: it is given to the previous function on the stack, until it reaches the function that contains the destination.

We shall emphasize that C-style returns are not compatible with assembly jumps: a C return statement would execute a postlude whose corresponding prelude have not been executed. We have to check that a function that is accessed with an assembly jump does not end with a C return. Likewise, the end of such functions should not be reachable as it would imply the execution of an implicit return, including the execution of the postlude. The control can only escape by a distant jump from a function accessed by a distant jump. Let us illustrate with example of Listing 17.4. We assume that the execution starts at f with a call stack that does not contain any of the functions of this snippet. We start by analyzing f. After line 18, the state is EAX → 0, other variables are not relevant for our example. Line 19 jumps to label g in function g_wrapper. Current state is saved and replaced by ⊥. This jump is detected to be a forward distant jump and thus we create the map g: (EAX → 0) of forward distant jumps. Then, the analysis of the current function proceeds with only ⊥ until the end. There is no map of local jumps, thus, there is no need to iterate the analysis of this function for now. But, there is a map for forward jump, thus we need to analyze new functions. Since the destination is in g_wrapper, we analyze this function. The current state at the beginning is ⊥, but it also gets the map g: (EAX → 0), which is a map for local jumps in this context. At this point, the call stack contains both f and g_wrapper.

During the analysis of g_wrapper, the current property stays ⊥ until we reach label g. When we do, the saved property is restored. After line 11, it is EAX → 1, and at line 12, we execute a new forward distant jump, thus we use the same strategy as for * * Indeed, we need such a lenient handling of control flow because it is hopeless to understand it statically. If we could know in advance which jumps are forward or return, we could infer interesting structures. Analysis of h_wrapper starts as the analysis of g_wrapper: ⊥ as current property and saved map h:(EAX → 1). The stack contains f, g_wrapper and h_wrapper. After line 3, the property is restored and the current state becomes EAX → 1, and EAX → 3 after line 4. Here, the destination of the jump is found in f, which is already in the stack. This jump is thus a return distant jump. The current property becomes and stays ⊥ until the end of the function. As there are no local jump maps, we reached the fixpoint, and since there are no forward distant jump maps, there is no new function to analyze. But the map r:(EAX → 3) is returned to the "calling" function (according to the stack): g_wrapper. Here, we detect that there is no new property as the return distant jump goes even further. No new analysis is required, and the map is forwarded even deeper. Function f gets in input both the initial environment and the return jump r:(EAX → 3) that is local in f's context. Its analysis gives the same result until line 20. Before, current property is ⊥, like it was at the first iteration, but now, there is a state to restore jumps, until we get a return distant jump. Listing 17.6 shows the remaining of this analysis, propagating backward the return distant jump.

220CHAPTER 17. CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR

We can easily see that this method works perfectly with C analysis as it does not interfere.

To formalize this method, we need a stack to distinguish forward and return distant jumps. We analyze a function, and we iterate it until we reach a fixpoint for local jumps. Then, we analyze function pointed by forward distant jumps (and recursively), and we collect return jumps. The function is then reanalyzed with these new jumps, until fixpoint of forward jumps. Each iteration to reach the fixpoint of forward jumps imply to find the fixpoint of local jumps and to analyze functions containing the destinations.

We need to adapt slightly the type D to allow maps for distant jumps:

D := D S × (L → D S ) × (L → D S ) × (L → D S ) × D S
Such objects contains respectively the current state, the map of local jumps, the map of forward distant jumps, the map of return distant jump and returned states. We need to adapt slightly the semantics of jumps so that it saves the current state to the right map. For this step, we need to be able to translate a code pointer to program points (this issue has already been discussed), to know in which function stands each program point, and to know whether each of these functions is in the call stack.

After stabilization of local jumps, the semantics of a function is a function of type

f : D L S → D L S × D L S × D S
where L is the set of assembly program points. The parameter is the map of new local jumps: distant jumps that are local in the context of the function. Returned values are respectively the map of forward distant jumps, the map of return distant jumps and the returned property.

From f , we build ←f the function of type

← - f : D L S → D L S × D S
that stabilized forward distant jumps and returns the return distant jump and C-returned property. For that, we get all bindings l → s (with s non-empty) in the forward distant jump map returned by f , and for each such binding, we use the function ←g where g is the function containing program point l, with the input map l → s. This recursive process terminates since the depth cannot be greater than the number of functions in the program. Indeed, when there is no forward distant jumps, we have

← - f (m) = (b, c) where (⊥, b, c) = f (m)
For functions accessed by distant jumps, such as g, we have to check that c = ⊥, and raise an alarm otherwise. In ←f , there are still some returned flows. There is no relevant notion such as the semantics of a function with such arbitrary jumps. We are instead interested in the 224CHAPTER 17. CONTROL STATEMENTS AND MODIFICATION OF THE ITERATOR semantics of the program. For that, we consider the function ←-main where main is the entry point of the program. As there is no caller for main, it cannot produce return distant jumps. Then, ←-main stabilized all jumps and, consequently, it is the semantics of the program.

In this method, we analyzed independently each jump l → s and m → t, even if program points l and m are located in the same C function, which allows a better precision. We could also analyze it once, by injecting the map l → s, m → t. This could allow better performances. In practice, the precise strategy was good enough in terms of execution time.

Mixed Calls

Mixed calls from assembly to C do not require more care than respecting the calling convention. Indeed, from the point of view of the control flow, there are just like regular C calls: the callee is a standard C function and should end with a normal C statement. Of course, during the execution of the callee, the control flow can be complex, but this is outside the scope of the mixed call.

But calls from C to assembly are more complicated. The assembly callee should return properly so that the C caller can continue correctly its execution. Calling convention cdecl specifies that the callee gets a stack containing (from top to bottom † † ) the return address and parameters, as shown on Figure 17.1. Return address R is a code pointer in compiled C code, just after the generated CALL instruction. According to the calling convention, return is expected to jump to address R with a stack that still contains parameters.

R

Return address param 1

param n

Parameters

Call frame of caller Figure 17.1: State of the stack just after a call from C to assembly † † That is, in increasing addresses. We recall that x86 stacks grow down in memory.

FAR JUMPS AND TASK SWITCHES

225

This case is not different from the other ones: RET instruction is not special here either. One could perform the same jump by popping the address in a register before using a simple JMP. But the return address is particular. We cannot tell its value, thus, we handle it purely symbolically. When jumping to R, the analyzer checks that the stack still contains n variables, and performs the jump just like a C return: EAX holds the returned value and the execution of the caller can resume. We cannot merge C returns with "jump to R" returns. Indeed, in this context, C returns are forbidden since the caller called directly an assembly label: we need to distinguish these two kinds of returns.

To handle symbolically R in Astrée, we push a special value on the stack that can be moved, but is never the result of any operation, which guarantee that if the top of the stack is this special value, it has not been modified. It fails to analyze a program that pops the return address, applies some computations that does not modify the final value (like an increment and a decrement), and pushes it back before returning. There is no such code in our target software and it was judged to be dubious, for the least. If we want to be able to analyze such code, we need to make R and ordinary unknown value, and keep a copy. Return is successful if we manage to prove that the current value is still the original value. Power of this method depend directly on the precision of underlying domains. The technique explained in Part IV "A Reduced Product with Ghost Variables" (page 229) might help to prove that the return address is correctly restored.

Return distant jumps are still possible and does not end the call.

Far Jumps and Task Switches

For now, we have explained how to make any near jump work with C code. This kind of jumps only updates EIP, the instruction pointer, and thus cannot switch between tasks, use a new segment register or, a fortiori, perform inter-privilege calls. These calls are very important during boot to set up segments, tasks. . . . They also happen when using a call gate or handling an exception (for a syscall, typically).

Fortunately, these jumps are not very different from near jumps. Pseudo-code given in [START_REF]Intel® 64 and IA-32 Architectures Software Developer's Manual[END_REF] shows that far jumps can be decomposed into multiple checks and assignments, and an update of EIP. This local view drops completely the intuitive notion of tasks or temporary privilege elevation. These are not intrinsic to instructions, but are emergent properties resulting of a good use of these features, just like the notion of procedure comes from a nice use of calls and returns. But as near jumps are arbitrary in assembly, and not well parenthesized, far jumps are sometimes used in surprising ways. It would be vain to try to give a high-level interpretation of far jumps, thus, we consider it as a sequence of assignments, checks and a simple jump.

In Astrée, far jumps have been quite easy to implement thanks to microcode (see subsection 16.1.3 "Microcode" (page 197)), by translating directly microcode in Intel's manual.

Introduction

S ome properties are easier to prove using quantities that do not appear directly in the semantics of the program. A semantics where such quantities are made explicit is called non-standard. It does not have more information, it is just presented differently, but it makes programs easier to analyze.

Using such implicit value is a common method in physics, since they allow seeing the problem with a different point of view, which may make it easier to solve. This is especially interesting to exhibit some invariant. For instance, in thermodynamics, Laplace's law states that, for an ideal gas, during an isentropic process, P V γ is constant (where P is the pressure, V is the volume and γ the adiabatic index). Even if this value does not naturally appear as a parameter of the system, it may help to find actually interesting parameters, in this case, P and V . There are many other conservation laws; among very fundamental such laws, we can mention the conservation of energy-mass, the conservation of electric charge or the conservation of color charge [START_REF] Greiner | Quantum Chromodynamics[END_REF]. They all introduce a non-elementary value (resp. the total quantity of energy, the sum of electric charges and the total color charge) that is constant and may help to find the value of elementary parameters.

In abstract interpretation, the mission of an abstract domain is to prove some kinds of properties, and it may need to make explicit some hidden quantities, like physicists do. These are called ghost variables. There are as many kinds of ghost variables than there are kinds of properties. Unlike the examples we gave previously, ghost variables are not necessarily constant: they may change or be refined during the analysis. For modularity reasons we need a way to make several domains to cooperate with a decentralized handling of ghost variables. This is the goal of this part.

In the following chapters, we will see how to make a product of abstract domains with ghost variables, examples of such domains and implementation details. This part is a detailed version of [START_REF] Chevalier | Sharing Ghost Variables in a Collection of Abstract Domains[END_REF]. In this chapter, we will see how ghost variables may be useful and why existing solution lacks generality.

Motivation

Firstly, let us illustrate a few examples where ghost variables are useful. They will help to point out difficulties and constraints that one should expect from a reduced product with shared ghost variables.

Pointers

This first example is not particularly linked to the support of mixed C and assembly: it is the abstraction of pointers. As said before, Astrée uses the abstraction described in [START_REF] Miné | Field-sensitive value analysis of embedded C programs with union types and pointer arithmetics[END_REF][START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF]. This method describes concrete pointers as an unevaluated sum of a base (a function, the beginning of an aggregate-typed variable (array or structure) or a scalar-typed variable) and an offset relative to this base. Abstract pointers are represented by a set of possible bases and an abstract integer. None of these quantities appear as it in the standard semantics, but we can make them explicit in an enriched semantics. The set of bases is represented directly, since it is not bigger than the set of all declared variables and functions, thus reasonably small. But the abstract integer is not as nice, we depute its representation to an abstract domain that is able to handle integer variables. In Astrée, this domain is the product of relational numerical domains.

The abstract domain that handles pointers is a parametric domain that accepts an underlying numerical domain (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)). All the commands that come to the numerical domain are generated by the pointer domain. When it needs to represent a pointer's offset, it commands the numerical domain to add a new variable to represent it. Then, all operations on pointers are translated into operations on offsets. For instance, in Listing 18.1, p initially points to x[0], its base is then x and the offset, denoted Offset (p) is 0. At line 3, p is incremented, according to pointer arithmetic, which is translated into Offset (p) ← Offset (p) + 1 × sizeof(int), which a purely numerical assignment. From the numerical domain's point of view, the offset is a regular variable, it has no idea that it is a ghost variable. This is very easy since the pointer domain is stacked over the numerical domain, it can rewrite all commands coming from above (struct domain) so that the numerical domain sees only the world built by the pointer domain. This stack structure is very typical, as it can be also found in Verasco [START_REF] Jourdan | A Formally-Verified C Static Analyzer[END_REF] and is the cornerstone of parametric domains such as power set completion or partitioning. Yet it implies a hierarchical ordering of domains: each domain control the underlying (composite) domain. This strategy does not allow a fully decentralized sharing of ghost variables.

This abstraction already exists in Astrée, but we want the new framework to be able to include an adaptation of this domain. By doing so, we want to get rid of the limitations coming from this hierarchy: for instance we would like the integer domain to be usable by any other domain.

GDT Entries

In subsubsection 2.3.2.2 "Protected Mode Segmentation" (page 19), we have explained segmentation in x86 architecture. It is a memory protection feature that allows cutting the memory into contiguous pieces called segments. Each segment is described by a base address, a length and some access flags. These pieces of information are stored in a memory structure called segment descriptor (see subsubsubsection 2.3.2.2.1 "Segment Descriptor" (page 19)), whose fields are arranged as shown in Figure 2.7 "Layout of a segment descriptor" (page 20).

Call gates are protected pointers to a more privileged procedure (see subsubsection 2.4.2.3 "Inter-Privilege Calls" (page 31)). They form, with similar features (trap gates, interrupt gates and task gates), a very useful way to allow controlled privilege transfer. These gates are described by structures very similar to segment descriptors. For instance, call gate structure is shown on Figure 2.13 "Structure of a 32-bit call gate" (page 31).

We can see that fields in segment and trap descriptors are not contiguous. For instance, in call gate descriptors, the pointer to the procedure is cut in 3 parts. They are usually built with bitwise computations, as shown on Listing 2.1 "Building a segment descriptor" (page 20). When these descriptors are used by the processor, it reconstructs the fields transparently. To analyze successfully such code, we need to be able to handle precisely bitwise operations on pointers. Especially, we want to prove that a pointer that is split into multiple parts and then rebuilt gets its original value back. For instance, in Listing 18.2, function f returns the same value it was given in argument, no matter if the value was an integer or a pointer. Lines 2 and 3 cut the argument using bitwise expressions, this is a simplified version of the construction of a segment/gate descriptor. Line 4 rebuilds the original value, this is what the processor does silently. In the case of a gate descriptor, int rebuilt is the reconstructed pointer to the desired procedure.

CHAPTER 18. INTRODUCTION

The problem posed by this kind of code can be easily solved using ghost variables. We need ghost variables to remember from which expression each bit-slice comes from. After line 2, int low is made of two parts: 16 lower bits are the 16 lower bits of int x, while 16 upper bits are zeroes. Zeroes can be stores as it, but we need a ghost variable to store the lower part. Indeed, we cannot just keep the relation "16 lower bits of x are the 16 lower bits of low" since it would not hold if x is modified and cannot be established if x is replaced by an rvalue, like x + 1 * . Instead, we use a ghost variable low which is assigned the value of x, and we remember the relation that "16 lower bits of low are the 16 lower bits of low". By doing so, we rely on another domain to remember than low = x, at least until x is modified, and then to keep a relation between them. The relation can be drawn as on Figure 18.1, using the notation of bit slice extraction as explained in subsection 2.1.3 "Conventions" (page 9) and section A.1 "Architecture" (page 369). For the sake of clarity, we need a most systematic nomenclature of these ghost variables. We name Slice [c:d] (v) is a good candidate as it allows easier deduction later. Once again, using another value is still correct, but subsequent computations and refinements may fail to give something interesting.

After line 3 of Listing 18.2, the 16 higher bits of high are zeroes and the 16 lower bits are the 16 higher bits of x. We introduce a variable Slice At line 4, the right-hand side expression is a 32-bit value whose 16 higher bits are the 16 higher bits of Slice Using this method, we can solve this kind of problems for any kind of variables that are representable in ghost variables. If we use a stack architecture, this domain should lie above any other domain that are used to handle the value of variables we slice. Here, we would need to put this domain above stack domain. For this example, it would work, but we will see later how this solution lacks generality.

Linear Combinations of Pointers

When writing low-level software, bit slicing is not the only kind of strange trick that we use. Here, we will see that linear combinations of pointers can be useful. In Listing 16.1 "Simple dynamic code with problematic handling" (page 201), we have shown a use of the difference of two pointers that does not point to the same block. This is really easy to solve by representing this value as an unevaluated linear combination. Each term is made of an integer coefficient and a ghost variable. This could be the job of a dedicated domain that handles linear combinations and which lies above the pointer domain, as term of the linear combination are also pointers.

We can think of an even more problematic case. We already talked about call gates. They allow unprivileged programs to access a privileged procedure. They contain a segment selector in which the procedure is stored and the offset inside the segment. In descriptors, most fields are not contiguous, in particular for call gates, the offset is indeed cut into two parts (see Figure 2.13 "Structure of a 32-bit call gate" (page 31)). Two problems are mixed here. First, the offset inside the segment is the difference between the procedure address and the base address of the segment: the offset is a linear combination of pointers. Then, this linear combination is split into two pieces, as explained in subsection 18.1.2 "GDT Entries" (page 231). To get a precise abstraction of this problem, we need a bit slice domain, that requires a linear combinations domain to represent sliced values, linear combinations domain needs a pointer domain to represent each term, and the pointer domain requires a numerical domain to handle offsets.

If we assume a stack architecture, domains should be ordered as cited, from top to bottom. But this is very constraining: we can only represent nesting of properties that follows this order. We can handle slices of integers, slices of pointers, linear combinations of pointers, slices of linear combinations of pointers. . . ; but we cannot represent linear combinations of slices of pointers, as it is not the order enforced by the stack.

Related Work

A very standard and generic way to improve the precision of an analysis is to use partitioning. The point is to represent the state as a disjunction of states that are easier to abstract precisely. In these works, ghost variables are states we use in the disjunction. States can be partitioned depending on their contents (as in [START_REF] Bourdoncle | Abstract interpretation by dynamic partitioning[END_REF]) or the context (as in [START_REF] Amato | Efficiently intertwining widening and narrowing[END_REF]).

Another kind of information that may be relevant for abstraction is the history of objects. This forms also a non-standard semantics since it keeps in the current state events that are in the past, and thus are only visible in traces according to the standard semantics. [Fer00; Ven98; Ven99] use such history. In [START_REF] Bor-Yuh | Abstract Interpretation with Alien Expressions and Heap Structures[END_REF], ghost variables are used to represent expressions with unevaluated external symbols.

In all these enriched semantics, the role of ghost variables is very static. [Pér10; HP08; AČW09] are examples of works where ghost variables have a more dynamic semantics, but they are local to the domain and are not shared with outside domains.

DIFFICULTIES

235

Non-standard semantics make some pieces of information explicit to ease the abstraction, but we still need domains to abstract precisely this newly explicit quantities. A way to improve precision of domains is to make them work together. We have already explained product of domains in chapter 12 "Product of Abstract Domains" (page 155) (see also [START_REF] Cousot | Systematic Design of Program Analysis Frameworks[END_REF][START_REF] Cortesi | A Survey on Product Operators in Abstract Interpretation[END_REF]), and how to make a realistic reduction operator in section 12.3 "Partially Reduced product" (page 159) (see also [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF]).

There are other works on domain cooperation with dynamic support, like cofibered domains [START_REF] Venet | Abstract cofibered domains: Application to the alias analysis of untyped programs[END_REF]. They have some limitations that we try to overcome. The product we propose does not enforce a hierarchy between domains and the current support is known by every domain to improve precision.

[JMO20] also uses transformation of statements but based on expression rewriting. In this approach, several abstraction levels use rewriting rules to gradually simplify expressions. This strategy is well-fitted for function resolution in a context where there are function overloading and dynamic typing, which is crucial to analyze Python source code. Given our application, we have other priorities. This is why we choose to promote a more flexible framework where statement transformation may depend on the history (see chapter 22 "Slices Domain" (page 281) for an example). It allows domains to declare new ghost variables without knowing a priori when they will be used; their value will be updated during the following computation steps, and used when useful without knowing precisely when they have been declared. Another requirement is to allow arbitrary predicate nesting (see section 18.1 "Motivation" (page 230)): domains are free to use ghost variables to represent any available property, even if it means that some kinds of transformations are not so straightforward.

Difficulties

We need a framework to build a composite domain that does not enforce a hierarchy. More precisely, that means that domains may allocate ghost variables dynamically and make all other domains know about these new variables. This way, ghost variables can be precisely abstracted by any involved domain.

Distributed handling of ghost variables implies that domains must communicate to know which variables are added, deleted, modified. . . . Each domain emits such constraints, but there is no way to order them meaningfully as there is no hierarchy. We need every ordering of these constraints to be correct.

This framework must perform reduction. In particular, with domains that need no ghost variables, it should behave just as the partially reduced product explained in [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF], and implemented in Astrée.

This framework should also make soundness and termination provable modularly. Domains must individually meet some constraints that guarantee soundness and termination of the whole composite domain. This is crucial both from a theoretical and an implementation point of view. Indeed, if each domain match local conditions, any assembly of them forms a composite domain that should be sound and terminating. This reduces the complexity of proofs and it should not be redone each time another set of domains is needed. We can have a library of domains and just select the ones we need to have required precision with the best performance. Similarly, each domain can be implemented independently, which is a nice property for software engineering.

The set of living ghost variables may depend on the history on the state. For instance, we need an offset for a variable if it was assigned a pointer value. Even states with the same concretization can use different sets of ghost variable depending on the way these states were obtained. During binary operations on states (like join), even if the language guarantees that the set of living (real) variables is the same in both operands (like C does), the set of ghost variables may be different. This means that we need to be able to perform binary operations with different set of living variables. The method that we will use is to start with a unification step before performing the binary operation on resulting states that have the same set of ghost variables.

Chapter 19

Concrete Framework

W hile termination is an intrinsic notion, soundness of an abstraction is relative to a concrete semantics. We explain here how to insert ghost variables in the concrete semantics. To be as general as possible, we will not use a precise language, but just define its semantics as a composition of opaque primitives.

Abstraction Level and Expressions

The use case of our new framework is to represent complex properties on pointers, and thus it will replace the pointer domain in the composite domain of Astrée (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)). This has some implications on the framework. In particular, we can ignore features that have been abstracted away in overlaying domains.

In particular, struct domain has already resolved pointer dereferences, thus expressions cannot contain indirections (C prefix operator *). Nevertheless, operator "address of" (C prefix operator &) is allowed: since the new framework is made for pointer abstraction, we should be able to build pointer values. We name V the set of variables and I a set of associated values. Concrete memory states are maps of type S := V I. The support of the map is the set of living variables. The set I shall contain all the values required to evaluate a variable in an expression. Given our abstraction level, it is usually a set of pairs made of the value of a variable and its address.

We should emphasize that variables are assumed not to alias from the point of view of pointer domain: assigning a variable never changes the value of another one. If some variables are used to represent the same memory location, it is the struct domain's responsibility to emit multiple instructions to update each variable corresponding to this location. For instance, in Listing 19.1, variable a can be represented in two ways: a single 4-byte variable (to store int x) or 4 1-byte variables (to store char y [START_REF]1 Dereferencing an arbitrary integer[END_REF]). We respectively name them x and y 0 , . . . , y 3 . The first assignment a.

y 1 ← x [8 : 15] y 2 ← x [16 : 23] y 3 ← x [24 : 31]
using notations for slices and hexadecimal values introduced in subsection 2.1.3 "Conventions" (page 9) and section A.1 "Architecture" (page 369). Then the second assignment a.y[0] = 0x02; is translated to

y 0 ← 02H x ← 02H + 2 8 y 1 + 2 2•8 y 2 + 2 3•8 y 3
The underlying domain receives 5 directives for the first assignment and 2 for the second, but from its point of view, y 0 , . . . , y 3 and x are independent.

Since variables do not alias, the independence of statements is easy to check. For instance, let x and y be variables, and e an expression if e does not contain x or y then assignments

x ← e y ← e can safely be commuted.

Since we want to stay fully general about the language, we will not define the set of expressions. We only assume that they are trees using a given set of constructors, where leaves may be variables.

Let h be a set of constructors * . Elements of h are written c /n where c is the name of the constructor and n ∈ N the arity.

ABSTRACTION LEVEL AND EXPRESSIONS

Notation 19.1 -Expressions

Let V be a set of variables. We denote E V the set of expression defined inductively with constructors h V where elements of V are used as niladic constructors.

The set V is arbitrary, and not necessarily a subset of V. In fact, it is useful with the set of real variables and with the set of ghost variables.

Notation 19.2 -Classical expressions

We denote

E := E V
the set of expressions with variables in V.

Later, we will need haunted (or ghost) expressions, that is expressions that involve ghost variables. The standard semantics involve only classical expressions. Haunted expressions will be useful in the enriched semantics, to express constraints on ghost variables.

Definition 19.1 -Structural inclusion

Let V be a set of variables and (e, e

) ∈ E V 2 . Let ij ⊆ E V 2 , the relation such that † : eije :⇔ ∃c /n ∈ h : ∃p ∈ 1, n : ∃(e i ) i∈ 1,n-1 ∈ E V n-1 : e = c /n (e 1 , .
. . , e p-1 , e, e p , . . . , e n-1 )

We write ⫑ the reflexive and transitive closure of ij. If e ⫑ e , we say that e is structurally included in e .

This means that e is a subtree of e .

Proposition 19.1 -Well-foundedness of structural inclusion

⫑ is a well-founded order relation.

Proof. ⫑ is a preorder as it is reflexive and transitive by definition.

Let (e, e ) ∈ E V 2 . We assume that e ⫑ e and e ⫑ e. If eija 1 ij • • • ija l ije then the height of the tree of e is greater than the height of e, which is not possible since e ⫑ e. Thus, both heights are equal, and e = e . The relation is thus antisymmetric, thus it is an order relation.

useful but quickly forgotten afterward, like Linear B alphabet was useful once, but gave nothing and was replaced by Greek alphabet. h is the character for syllable "ko" or "co", like "constructors". Another benefit of Linear B characters is that they have very little background in notations, while most Greek and Hebrew letters are heavily used and almost have intrinsic meaning, now.

† The symbol ij is also a Linear B character (whose shape is explicit enough for a transition relation), consequently, we will not speak about it again.

CHAPTER 19. CONCRETE FRAMEWORK

Let (e i ) i∈N ∈ E V N . If e i is decreasing (according to ⫑), so is the sequence of heights. Since (N, ) is well-founded, so is (E V , ⫑).

Notation 19.3 -Variables in an expression

Let V be a set of variables and e ∈ E V . We write Var (e) ⊆ V the set of variables in e, that is

Var : E V → P (V ) e →    n i=1 Var (e i ) if e = c /n (e 1 , . . . , e n ) {e} if e ∈ V
Usually, we would define the semantics of expressions by induction over the structure. This is not necessary here, we only want the result of an affectation to update the lefthand side variable only depending on the variables in the right-hand side expression. This way, we do not even need the semantics of expressions to be compositional.

The other fundamental objects built from variables are conditions. Indeed, without conditions, a programming language is no more than a linear sequence of assignments, which is not very expressive. For the sake of simplicity, we only use conditions made of a comparison operator (with arbitrary arity) and expressions.

Notation 19.4 -Conditions

Let O be a set of comparison operators. For all set of variables V , we denote

C V := op /n (e 1 , . . . , e n ) op /n ∈ O, (e i ) i∈ 1,n ∈ E V n and C := C V O is usually = /2 , = /2 , < /2 , /2 , > /2 , /2
. They are often used in infix notation, while omitting the arity.

Notation 19.5 -Variables in a condition

Let V a set of variables. For c = op /n (e 1 , . . . , e n ) ∈ C V , we write Var (c) ⊆ V the set of variables that appear in c, i.e.

Semantics

We assume the concrete semantics can be built by composing the following primitives:

-Alloc: allocates a variable, -Kill: removes a variable, -Assign: assigns a variable the result of an expression, -Guard: enforces a condition.

1 if non_deterministic(): 2 x = 1 3 y = 2 4 else: 5 x = 2
Listing 19.2: A set of living variables that cannot be statically inferred Despite our quest of generality, we need some hypotheses on the language and on primitives. The first important point to notice is that the support (i.e. set of living variables) is dynamic. We recall that deterministic memory states are elements of S := V I. But in a non-deterministic semantics, we might be tempted to use P (S), but this allows states containing maps with different support. This exists in some programming language with very few static guarantees, like Python. For instance, in Listing 19.2, after line 5, the non-deterministic memory state is

(x → 1, y → 2) (x → 2)
For the sake of simplicity, and because we are interested in C and similar languages, we restrict the set of non-deterministic states to sets in which all elements have the same support:

D := S ∈ P (S) ∀(s, s ) ∈ S 2 , supp (s) = supp s Notation 19.6 -Support
For all a ∈ D, we write supp (a) the common support of all maps in a.

Before giving the signature of primitives, we need another last notation.

Notation 19.7 -Assignments

Let V be a set of variables. We denote

A V := V × E V and A := A V CHAPTER 19. CONCRETE FRAMEWORK
The types of primitives are:

-

Alloc : V × D → D -Kill : V × D → D -Assign : A × D → D -Guard : C × D → D
We will give the hypotheses that we need on these primitives by category. First, let us see the support-related assumptions.

Informally, we expect Alloc to add a variable to the support, and Kill to remove a variable. The other primitives should not change the support.

Axiom 19.1 -Effect of primitives on the support

-∀v ∈ V, ∀d ∈ D, supp (Alloc (v, d)) = supp (d) ∪ {v} -∀v ∈ V, ∀d ∈ D, supp (Kill (v, d)) = supp (d) \ {v} -∀a ∈ A, ∀c ∈ C, ∀d ∈ D, supp (Assign (a, d)) = supp (d) -∀a ∈ A, ∀c ∈ C, ∀d ∈ D, supp (Guard (c, d)) = supp (d)
We also need to assume that binary operations, like unions at the end of branching, are performed with operands with the same support and that variables that appear in assignments and guards are all living at this point.

The second class of hypotheses is about variables that are read or written by each primitive. We implicitly lift notations on functions on elements of D. For instance, given d ∈ D and V ⊆ V, we write

d |v for f |v f ∈ d , and d [a → b] for {f [a → b] | f ∈ d}.

Axiom 19.2 -Variable allocation and killing

-∀v ∈ V, ∀d ∈ D, v ∈ supp (d) ⇒ Alloc (v, d) = d -∀v ∈ V, ∀d ∈ D, v ∈ supp (d) ⇒ Alloc (v, d) |supp(d) = d -∀v ∈ V, ∀d ∈ D, v ∈ supp (d) ⇒ Kill (v, d) = d -∀v ∈ V, ∀d ∈ D, v ∈ supp (d) ⇒ Kill (v, d) = d |supp(d)\{v}
We need primitives to be ∪-preserving. This implies that they are lifts of primitives whose second parameter has type S rather than D. This condition makes following axioms more powerful as they should stand on each map of any state of D.

Axiom 19.3 -Primitives are ∪-morphisms

Let I be a set. Let (d i ) i∈I ∈ D I . -∀v ∈ V, Alloc v, i∈I d i = i∈I Alloc (v, d i ) -∀v ∈ V, Kill v, i∈I d i = i∈I Kill (v, d i ) -∀v ∈ V, ∀e ∈ E, Assign (v, e), i∈I d i = i∈I Assign ((v, e), d i ) 19.2. SEMANTICS 243 -∀c ∈ C, Guard c, i∈I d i = i∈I Guard (c, d i )
Following requirements are common to all reasonable languages. They specify that the only variables read or written are those that appear in expressions. This is possible since there is no aliasing and no dereference.

∀v ∈ V, ∀e ∈ E, ∀d ∈ D, ∀V ⊆ supp (d) \ (Var (e) ∪ {v}), ∀(a i ) i∈V ∈ I V , ({v}∪Var (e) ⊆ supp (d)) ⇒ Assign ((v, e), d) |{v} = Assign (v, e), d [i → a i ] i∈V |{v}
∀c ∈ C, ∀d ∈ C, ∀V ⊆ supp (d) \ Var (e), ∀(a i ) i∈V ∈ I V , Var (c) ⊆ supp (d) ⇒ Guard (c, d) [i → a i ] i∈V = Guard c, d [i → a i ] i∈V
Domains communicate constraints about variables in an arbitrary order. Since this order has nothing to do with the semantics, we need our construction to be correct independently of the chosen order. Since constraints are enforced using primitives, we need to be able to commute them.

Axiom 19.6 -Commutation of Assign ∀a ∈ D, ∀(e, f ) ∈ E 2 , ∀(v, w) ∈ V 2 , v = w ∧ {v, x} ∩ (Var (e) ∩ Var (f )) = ∅ ⇒ Assign ((v, e), Assign ((w, f ), a)) = Assign ((w, f ), Assign ((v, e), a)) Axiom 19.7 -Commutation of Guard ∀d ∈ D, ∀(c, d) ∈ C 2 , Guard (c, Guard (d, a)) = Guard (d, Guard (c, a)) Axiom 19.8 -Commutation of Assign and Guard ∀a ∈ D, ∀e ∈ E, ∀op /n ∈ O, ∀(f i ) i∈ 1,n ∈ E n , ∀v ∈ V, v ∈ Var (e) ∪ n i=1 f i ⇒ Assign (v, e), Guard op /n (f 1 , . . . , f n ) , a = Guard op /n (f 1 , . . . , f n ) , Assign ((v, e), a)
These assumptions are quite reasonable and allow commuting statements as long as they are independent enough. Once again, we rely strongly on the fact that variables do not alias each other. Otherwise, these syntactic conditions would not have been strong enough.

For the abstraction, we will need a less common assumption that does not restrict the expressive power.

Assumption 19.1 -Restriction on assignments

We assume that any assignment, the left-hand side does not appear in the righthand side.

For instance, this assumption prevents to run x = x + 1;. But there is a trivial workaround: y = x + 1; x = y;, where y is a helper variable used only for such assignment. A fresh variable can be picked each time as an extra guarantee of non-interference through this intermediate variable.

Ghost Variables

Ghost variables are used to help to abstract other variables. In subsection 18.1.2 "GDT Entries" (page 231), we have illustrated how to use ghost variables to handle sliced variables. In Listing 18.2 "A complicated way to do nothing" (page 231), we have seen that we needed one ghost variable for low and one for high. But if x has a pointer value, so are ghost variables Slice [0:15] (high). Thus, these three variables need ghost variables to store their offsets: more generally, even ghost variables may need other ghost variables. In that goal, we define ghost variables inductively: each ghost variable is made of a constructor and a variable. The variable is the one that the ghost variable help to represent, and the constructor explains the relation between them (what the ghost variable means). For instance, given a variable x, Offset (x) is a variable representing the offset of x: Offset explains the relation between Offset (x) and x: the former is the offset of the latter. [c:c+b-a] (for all a ∈ 0, 31 , b ∈ a, 31 and c ∈ 0, 31 -b + a ) are roles. The name "role" is because these constructors hold the meaning of ghost variables we build with them. From now on, everything written in this Round Hand style (like O, R or S ) is about ghost variables and haunted objects (everything that contains ghost variables).

Notation 19.8 -Ghost variables

We denote by V the inductively defined set of all variables, starting with real variables (in V). Formally, for all i ∈ N,

V i := V if i = 0 {R(v) | R ∈ R, v ∈ V i-1 } otherwise and V := i∈N V i
Elements of V are called ghost variables.

Notation 19.9 -Ghost expressions, conditions and assignments

E := E V C := C V A := A V Definition 19.3 -Ghostly ordering Let ⊆ V 2 such that ∀(v, w) ∈ V 2 , v w ⇔ ∃n ∈ N * : ∃(R i ) i∈ 1,n ∈ R n : v = R 1 (. . . R n (w) . . .)
Let be the reflexive closure of , i.e.

∀(v, w) ∈ V 2 , v w ⇔ ∃n ∈ N : ∃(R i ) i∈ 1,n ∈ R n : v = R 1 (. . . R n (w) . . .)
If x y, we say that x is less real, or ghostlier, than y.

We extend this relation to

E 2 by ∀(e, e ) ∈ E 2 , ∀v ∈ Var e , ∃v ∈ Var (e) : v v ⇔ e e
That is every variable in e is ghostlier than a variable in e.

Proposition 19.2

Relation is an order on V .

Proof. We can see a ghost variable as a list of roles and a real variable. Given two variables v and w, v w if they share the same real variable and if the list of roles of w is a suffix of the list of roles of v. From this interpretation, the result is obvious:

-reflexivity: a list is a suffix of itself; -antisymmetry: if each list is a suffix of the other, they are equal; -transitivity: if a list is a suffix of a second and the second is a suffix of a third, then the first list is a suffix of the third.

Proposition 19.3

Relation is a preorder on E .

Proof. Given expressions e and e , e e if e is made of variables that appear in e of variables ghostlier than the ones that appear in e. Reflexivity is thus obvious: e is made of variables contained in e.

Transitivity is hardly more complicated. Let e a third expression and let us assume that e e and e e, then variables of e are ghostlier than variables in e and, a fortiori, variables ghostlier than variables of e are ghostlier than variables of e. Thus, variables in e are ghostlier than variables of e, hence e e.

Ghost variables have a meaning: we introduced them to keep some quantities and this should be enforced.

Definition 19.4 -Semantics of ghost variables

Let ❲_❳ : (V \ V) → P (V I) be the semantics of ghost variables, such that

∀v ∈ V , ∀R ∈ R, ❲R(v)❳ = s ∈ S ∃t ∈ ❲R(v)❳ : s |{r(v) | r∈R} = t |{r(v) | r∈R}
The inclusion is obvious, we just need to take s = t, but the converse inclusion is not. It means that the semantics of a ghost variable is not sensitive to the value of non-parent or non-sibling variables. If a state is in the semantics of a ghost variable, changing such an unrelated variable gives a state that should still be in the semantics. In other words, the semantics of a ghost variable only involves this variable, its immediate parent and siblings. As a direct consequence, the value of a real variable constrains only ghost variables under it. For instance, the semantics of Slice [c:d] (x) are indeed bits c to d of variable x. The semantics is usually carried by the outermost role: it may be more general, but in this case, the following may not be easy to implement in the framework we will introduce.

GHOST CONSTRAINTS

247

Each domain brings its own set of roles and, to keep modularity, it should not depend on roles defined by other domains.

As we add new variables, we need to take them into account in the memory state. Thus, we extend S so that its domain contains ghost variables. We need to add a restriction: if a ghost variable is living, all less ghostly (more real) variables are living too.

S := s : V I ∀v ∈ V , v ∈ supp (s) ⇒ v ↑ ⊆ supp (s) ∀v ∈ supp (s) \ V, s ∈ ❲v❳
where v ↑ is the set of variables greater than v with respect to (see notation 11.2 (page 138) and section A.3 "Set Theory" (page 371)). Likewise, we need to extend D. We have explained that elements of D must be set of maps with the same support. Here, this condition is slightly relaxed as we only require that all maps must have the same set of living real variables. Yet, maps contained in a single element of D may have different sets of living ghost variables.

D := S ∈ P (S) ∀(s, s ) ∈ S 2 , supp (s) ∩ V = supp s ∩ V
We assume that we can naturally extend primitives to ghost variables, including their axioms. The types of primitives become:

-

Alloc : V × D → D -Kill : V × D → D -Assign : A × D → D -Guard : C × D → D

Ghost Constraints

Values of ghost variables are determined by their semantics and values of real variables.

In our framework, for modularity reasons, each domain can ask to create new ghost variables. Then, each ghost variable is managed by the domain that created it: we assume this is the only domain that knows the semantics of the ghost variable. Since other domains do not know the semantics of ghost variables that come from this domain, they would be supremely imprecise about the value of this ghost variable (they will default to ). In this case, it would be useless to share its ghost variables with other domains as they cannot say non-trivial properties about them. To do better, domains must communicate information about ghost variables.

We will now take a look at the constraints exchanged between domains. The first, obvious, kind of constraints are simply conditions as they express relations between variables. The second kind of constraints are elements of A , but with a slightly different interpretation. The ordered pair (x, e) should be interpreted as the comparison x = e, but with a restriction: when this constraint is used x should be perfectly unknown ( ) and x ∈ Var (e). In this context, guarding by the condition x = e is just as the assignment x ← e. They are called directed constraints. This is interesting because running an assignment is much simpler and faster than enforcing an arbitrary comparison. This -N is an arbitrary finite set of nodes, -E ⊆ N 2 is the set of edges, -∃r : We can remark that constraint-graphs are always connected since there is exactly one source and one sink. Moreover, we can see that a constraint graph cannot be empty (for the same reason), and it cannot have a single node, since the source always has a successor and the sink always has a predecessor. We can still build a constraintgraph that does nothing by using 2 linked nodes, and an empty map u, as shown on Figure 19.5. Such a trivial graph is useful when there is no ghost variable to constraint. They especially come from domains that does not create ghost variable and thus have nothing to refine, for instance, see section 23.1 "Plain Old Numeric Domain" (page 293). Given c ∈ U and g = (N, E, u) ∈ G , we write

N → N : ∀(a, b) ∈ E, r(a) < r(b) (graph is acyclic), -∃!s ∈ N : (∀(a, b) ∈ E, b = s) (there is exactly one source), -∀s ∈ N, (∀(a, b) ∈ E, b = s) ⇒ (∃(a, b) ∈ E : s = a) (
c ∈ g :⇔ ∃v ∈ V : u(v) = c
This notation allows easily writing that an elementary ghost constraint appears in a constraint-graph.

Definition 19.7 -Semantics of a constraint-DAG

Let g = (N, E, u) ∈ G be a constraint graph. Given n ∈ N , we define ⟪g⟫ n the semantics of the node n as

⟪g⟫ n : D → D a →              u(n) p∈{p∈N | (p,n)∈E} ⟪g⟫ p (a) if n = ⟜ g ∧ n ∈ supp (u) p∈{p∈N | (p,n)∈E} ⟪g⟫ p (a) if n = ⟜ g ∧ n ∈ supp (u) a if n = ⟜ g
We define ⟪g⟫ : D → D the semantics of the graph g as

⟪g⟫ := ⟪g⟫ ⊸ g
Proof. This semantics is well-defined since the graph is acyclic. This can be formally stated using the map r that appears in definition 19.6 "Constraint-graph" (page 250).

The semantics of a constraint-DAG is computed by guarding by the condition of each node, and computing the union of properties from converging edges.

Definition 19.8 -Constraint-DAG soundness A constraint-graph g ∈ G is said to be sound if ∀a ∈ D,    s ∈ S s ∈ a ∧ s ∈ v∈supp(a)\V ❲v❳    ⊆ ⟪g⟫ (a)
In other words, when one uses the constraint defined by the graph, we exclude no legal environments: all memory states that match the semantics of ghost variables are kept.

Though any sound graph can be used, there are several ways to build them systematically. Here is a non-exhaustive list that covers most common cases.

-If we test equality between variables x and y in a context where both R(x) and R(y) exist, for a given role R, and the implication

x = y ⇒ f (R(x)) = g(R(y))
is true in all states (with f and g two functions simple enough to be expressed as an expression in the language), we can generate the one-node graph containing the unique constraint f (R(x)) = g(R(y)). If several such implications hold, we may simply sequence all graphs in an arbitrary order (since these conditions do not interfere). This is the general case of the example x = y (Figure 19.2). -To test difference between variables x and y in a context where (R i (x)) i∈ 1,n and (R i (y)) i∈ 1,n exist, for a given family of roles (R i ) i∈ 1,n and the implication

x = y ⇒ i∈ 1,n f i (R i (x)) = g i (R i (y)
) holds in any state, we can generate the graph that joins the results of these n conditions taken separately. This is like the comparison x = y (Figure 19.3). -Some constraints come from the language semantics but can be reinterpreted with ghost variables. E.g. since the ghost variable offset must coincide with the C standard offset, the latter can be substituted by the former. Now we have ghost variables and tools to work on them, we can use all that to build cooperating abstract domains with ghost variables.

Chapter 20

Generic Abstract Domains and Reduced Product

B uilding cooperating domains that use ghost variables is not a straightforward ab- straction of the concrete framework described previously. We need to make domains so that we can easily combine them, while keeping distributed the burden of ghost variable management and proofs of soundness and termination.

We will first exhibit the signature of such domains with required assumptions. In next sections, we will see how we can use components of this signature to make them cooperate in a correct and terminating way.

Generic Abstract Domain

A Detailed Execution of an Example with Ghost Variables

In this chapter, we will study an example that sublimates the difficulties we highlighted, while being short and quite simple. It is given on Listing 20.1. This snippet shows how to cut and recombine a pointer using bitwise operations. At the end, we want to show that x = y. We have already seen how ghost variables help to solve this kind of examples, but we have not shown how to do it with separate domains. Listing 20.1: A simple example that needs ghost variables Since the management of ghost variables is distributed, we have to associate each of them with a single domain that will have the right to create and delete them, and which is the only one that is assumed to know the semantics. There is a formal rub: we 254CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT need this belonging relation in the definition of domains, but we need domains to define belonging. To break the cycle, since we are not yet able to define the signature of an abstract domain, we introduce domain names.

Definition 20.1 -Domain names

Let N a set of domain names.

For the purpose of Listing 20.1, we need 3 domains:

-equality, because we aim to prove an equality property; -base-offset pointers, to represent pointer values; -slices domain, to handle bitwise operations. Thus, we assume N contains the names of these domains. We will call them respectively Equality, Offset and Slice. For this example, we can indifferently assume that N is restricted to these only 3 names, or contains additional useless names.

Of course, when using several domains, they should have distinct names.

Definition 20.2 -Owner of a role

We define o : R → N the map that, given a role, returns the domain to which it belongs. Given R ∈ R and d ∈ N, when o (R) = d, we say that d is the owner of R, or that R belongs to domains d. We write R d. We extend this notion to variables:

∀v ∈ V , ∀R ∈ R, R(v) o (R)
The belonging relation is purely syntactic: the outermost role is enough to decide the owner of a ghost variable. Real variables are not owned. We can also emphasize that there is only one owner for each ghost variable (i.e. variables of V \ V). We will detail the execution of line 4. Since this statement demonstrate all features the first three lines need, we will not show detail execution of them. Using our three This domain has no ghost variables and does not emit any reduction, the computation stops here.

Domain Slice owns all roles of the form Slice

To run this statement, domains have created this new tree of variables:

y Slice

V 0 = V V 1 V 2
This state is correct but not interesting. We now enter the second phase, which is regular reduction just as in [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF]. Thanks to Equality domain, Offset domain can infer that Slice and communicates it to other domains. That is bits 0 to 15 of y are bits 0 to 15 of x, and bits 16 to 31 of y are bits 16 to 31 of y. From these equalities, Slice domain can deduce that x = y, which is the expected result.

This illustrates why we need distributed handling of ghost variables and that why they should be known by other domains. In fact, when we use a ghost variable, it is likely that another domain can do better about it, otherwise we would not need a ghost variable. For instance, parts of low, high and y are pointers: it is more suitable to ask the Offset domain to represent them. Likewise, the offset is a numerical value so, while the Offset domain associates it to the pointer of which it is the offset, it is worth to let a numerical domain represents the offset value. But, if we know for offset variables, in general we do not a priori know the kind of value the ghost variable stores, for instance, in the case of bitwise slices that can be pointers, numerical values or anything else. Furthermore, we do not know in advance the list of available domains, so we cannot decide which domain is the best to represent a ghost variable. Thus, every ghost variable must be known by all domains.

Let us discuss the problems that we have highlighted. The termination is satisfied: directed reductions of offsets do not trigger new constraints. We can observe that each constraint is more elementary than the previous, and there is nothing simpler than copying a numerical variable. The decreasing complexity is a good approach to ensures termination. The other problem was the execution order. Here, we can see that the assignments (from directed constraints) are disjoint: we assign variables under y from variables under low and high. So, an assigned variable is never in the right-hand side of an assignment and since we always go deeper in ghost variables, each variable is assigned only once, making directed reductions legal. Both these points will be detailed and generalized later.

258CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT

Generic Abstract Domain Signature

We can be very general on the form of the abstract domains. Classically, a domain needs a set of abstract states and their meaning in the concrete world, fixpoint approximation and abstract counterparts of concrete primitives. In addition, we add two other kinds of function.

First, there are two maps to decide what to do with ghost variables: one to react to the execution of a unary statement or a ghost constraint, and one to unify supports before performing a binary operation (typically, the join).

Moreover, domains include primitives to communicate information about their abstract state à la [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF]. This allows domains to refine themselves (as a reduced product is meant to) but also to communicate their policy on ghost variables management: since all domains must care about the ghost variables of the other domains, they need to communicate what to do.

We are given a lattice IO with a concretization γ IO : IO → D. This is a lattice common to all domains that will be the middleman for all communication.

Definition 20.3 -Generic abstract domain

A generic abstract domain with dynamic support is a tuple n, D , γ, , lfp , Assign, Guard, Alloc, Kill, Extract, Refine, ⪡, U, B where:

-n ∈ N is the domain name, -D is the set of abstract properties, -γ :

D → D is the concretization, -: D × D → D -lfp : (D → D ) → (D → D ) -Assign : A × D → D -Guard : C × D → D -Alloc : V × D → D -Kill : V × D → D -Extract : IO × D → IO -Refine : IO × D → D -⪡⊆ U × U -U : U × P (U ) × D → P (V ) × P (V ) × G × P (U ) -B : N * × D × D → (P (V ) × P (V ) × G ) 2
Assumptions that makes the domain correct and terminating are quite long and cannot be decently given in a single definition. They will be exposed in the following of this subsection.

To avoid redundant redundancy, all the following should be understood in the context

∀(a, b) ∈ D 2 , γ(a) ∪ γ(b) ⊆ γ(a b)
This means is a sound approximation of ∪ this is useful, for instance, at the end of if-then-else structures.

Assumption 20.2 -Soundness of lfp

∀f : D → D, ∀c ∈ D, ∀f : D → D , ∀a ∈ D , (f • γ ⊆ γ • f ∧ c ⊆ γ(a)) ⇒ lfp c f ⊆ γ(lfp a f )
Usually, to find a sound approximation of the concrete least fixpoint, we use a widening operator. Though, there are many strategies to increase precision, quite general ones (delayed widening, narrowing) or very ad hoc heuristics. The way by which we find this fixpoint approximation is not important here. By using a lfp operator, we are the most general. Each implementation of a generic domain can use its own strategy.

Assumption 20.3 -Soundness of Assign ∀(v, e) ∈ A , ∀a ∈ D , Assign ((v, e), γ(a)) ⊆ γ(Assign ((v, e), a)) ∧ let d := γ(Assign ((v, e), a)) in ∀(w, s, x) ∈ v ↓ × d × I, s [w → x] ∈ d
There are two parts here: when there is an assignment, the assigned variable is indeed modified accordingly. But old values of ghostlier variables may not be correct anymore. To be sound, Assign should set all ghostlier variables than the assigned one to .

Another point of view on assignment (v, e) is to decompose it into two successive computations. The first sets all variables of v ↓ to . The second is to apply the directed constraint v ← e, that refines the value of variable v.

Following assumptions are simply the soundness of various components of an abstract domain. There are no subtleties here. These functions are used to perform communications across domains, as done in [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF]. The next assumption is about U. This function is quite special, as it is the core of ghost variable handling: given a ghost constraint, it yields sets of variables to allocate and kill, and it generates new constraints (as a constraint-DAG). We must ensure that this graph is sound. Moreover, since each constraint generates new constraints, we need assumptions to enforce termination. Lastly, there is the problem of directed reductions: one can emit a directed reduction x ← e only if x is . This is guaranteed by some requirements on U.

Before giving this big list of hypotheses, let us define a relation of U. If u n v, then the domain of name n ∈ N is allowed to produce constraint v given the constraint u. This relation is useful for soundness, but it is not enough to ensure termination.

Definition 20.4

Let n ∈ N. Let n be the smallest relation on U satisfying:

-

∀v ∈ V , ∀R ∈ R, ∀(e, e ) ∈ E 2 , R n ∧ e e ⇒ (v, e) n (R(v), e ) -∀ P /p , Q /q ∈ O 2 , ∀(a i ) i∈ 1,p ∈ E p , ∀(b i ) i∈ 1,q ∈ E q , p q ∧ ∀i ∈ 1, q , ∃j ∈ 1, p , b i a j ⇒ P /p (a 1 , . . . , a p ) n Q /q (b 1 , . . . , b q ) -∀P /p ∈ O, ∀e ∈ E , ∀(a i ) i∈ 1,p ∈ E p , ∀v ∈ V , (∀i ∈ 1, p , a i e) ⇒ (v, e) n P /p (a 1 , . . . , a p )
This definition is quite austere, let us rewrite the three cases:

GENERIC ABSTRACT DOMAIN

261

-given a directed constraint, one can generate a directed constraint on an immediately ghostlier variable, using a ghostlier expression; -given a condition, one can generate another condition using only ghostlier expressions; -given a directed constraint, one can generate a condition using expressions that are ghostlier than the right-hand side of the directed constraint. All these cases induce increasing ghostliness. From a directed constraint x ← e, we can generate directed constraints y ← f where y is ghostlier than x and f ghostlier than e, and thanks to assumption 19.1 "Restriction on assignments" (page 244) no left-hand side of such directed constraint can appear in right-hand sides. This is a crucial argument in following proofs. Relation ⪡ is a well-founded order on U .

Assumption 20.10 -Hypotheses on constraint triggering

∀a ∈ D , ∀u ∈ U , ∀U ∈ P (U ) , let (new, old, g, L) := U(u, U, a) in let (N, E, f ) := g in a) new ∩ supp (a) = ∅ b) old ⊆ supp (a) c) ∀c ∈ g, c ∈ A ⇒ π 1 (c) ∈ supp (a) ∪ new ∧ Var (π 2 (c)) ⊆ supp (a) d) ∀c ∈ g, c ∈ C ⇒ Var (c) ∈ supp (a) ∪ new e) g is sound, f ) ∀v ∈ g, u n v g) ∀(i, j) ∈ N 2 , f (i) ∈ A ∧ f (j) ∈ A ∧ π 1 (f (i)) = π 1 (f (j)) ⇒ i = j h) ∀v ∈ g, v ∈ U i) L = U ∩ u ↓⪡
This is quite a long assumption. Items a and b mean that we allocate variables that were not alive and kill only living variables. Items c and d are sanity hypotheses: constraints only involve living or new variables.

Following items are less obvious and more interesting. Item e ensures that we will only refine the abstract state with respect to ghost the semantics of ghost variables. Items f and g are useful to prove that we use correctly directed constraints: the former means that a domain trigger only ghostlier constraints, and the latter means that in a graph, there cannot be two directed constraints about the same variable. Items h and i are useful to ensure termination. Termination criterion may seem more flexible, but 262CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT in practice, it is not that much. This is in fact the least modular point. This will be explained in section 20.4 "Termination" (page 266).

The last component of generic abstract domains is the function

B : N × P (U ) × D × D → (P (V ) × P (V ) × G ) 2 × P (U )
Its purpose is to unify the support of two abstract states, in order to perform a binary operation.

Here, we do not want to refine the state, thus we can use simpler assumption: we only need the result to be bigger. To be able to compute the result, we introduce the abstract semantics of constraint-graphs and graphs with set of variables to add and kill.

Definition 20.5 -Abstract semantics of a constraint-DAG

Let g ∈ G be a constraint graph. We define ⟪g⟫ : D → D the abstract semantics of the graph g.

Definition 20.6

Let (new, old, g)

∈ V × V × G . We define ⦉(new, old, g)⦊ : D → D by ⦉(new, old, g)⦊ = v∈old Kill (v, _) • ⟪g⟫ • v∈new Alloc (v, _)
where _ is a placeholder (see section A.5 "Function Theory" (page 374)).

A 3-tuple made of sets of variables to allocate and kill and a constraint-graph is called an enriched graph.

Assumption 20.11 -Assumptions on support unification function

∀(a, b) ∈ D 2 , ∀i ∈ N * , let (G a , G b ) := B(i, a, b) in let a := ⦉G a ⦊ (a) in let b := ⦉G b ⦊ (b) in let (new a , old a , g a ) := G a in let (new b , old b , g b ) := G b in let V := {R(v) ∈ V \ V | v ∈ V , R ∈ R, R n} in a) new a ∩ supp (a) = ∅ b) old a ⊆ supp (a) c) new b ∩ supp (b) = ∅ d) old b ⊆ supp (b) e) γ(a) ⊆ γ(a ) f ) γ(b) ⊆ γ(b ) g) (∀j ∈ 0, i -1 , supp (a) ∩ V j ∩ V = supp (b) ∩ V j ∩ V) ⇒ (∀j ∈ 0, i -1 , supp (a) ∩ V j ∩ V = supp (a ) ∩ V j ∩ V = supp (b ) ∩ V j ∩ V ∧ 20.2. PRODUCT OF DOMAINS 263 supp (a ) ∩ V i ∩ V = supp (b ) ∩ V i ) ∩ V
Here, G a is an enriched graph to apply on a, and G b is to apply on b. The integer i stands for the level of ghost variables we want to unify. By performing successive calls to B with increasing i, we unify the whole support.

Items a to d are similar to the first two of assumption 20.10. Items e and f ensure that this unification process is sound by being an overapproximation of the identity. Item g is the interesting point: if the support of a and b, restricted to V 0 ∪ • • • ∪ V i-1 and to variables belonging to the domain of name n, are equal, then B provides operations that does not change this part of the support, but also unify the i th level of ghost variables that belongs to the domain. Using the B function of each domain, we unify the whole i th level of ghost variables.

Product of Domains

Now we have seen the whole signature of a generic domain, with assumptions, let us explain how to use all these functions to build a composite domain that handles shared ghost variables. In this section, we are only interested in the working, how to connect everything. Correctness and termination will be explained later. We will also see later how to handle binary operations.

Given a constraint, function U provides new variables to allocate, variables to kill and more constraints about ghost variables, according to their semantics. The fourth returned value is a set of constraints that can be triggered by recursive calls to U. This is useful to ensure termination.

To get the better precision, we shall call U recursively to get even more constraints. We recall that U has type U × P (U ) × D → P (V ) × P (V ) × G × P (U ). Let a ∈ D be the abstract state we want to refine. Let N ⊆ N be the set of the names of abstract domains in the product. We let (U n ) n∈N the "U map" of each domain. Function

U n a : U × P (U ) → P (V ) × P (V ) × G × P (U ) (u, U ) → U n (u, U, a)
where n ∈ N , is the partial application of U n where the third argument is a.

We want to combine all functions U n a in a single function U a that collects constraints from all domains, and, from that, we build a function U * a that performs this operation recursively. The first step into this direction is to share all U n a functions between all domains. This can be achieved through Extract and Refine. Given a communication domain IO and an abstract domain D , we can enrich them into

IO := IO × P U × P (U ) × D → P (V ) × P (V ) × G × P (U )       let g := U a (u, U ) in lfp g         (N, O, G, U ) → let R := U a (u , U ) ∃g ∈ G : u ∈ g ∈ G in      N ∪ {π 1 (r) | r ∈ R}, O ∪ {π 2 (r) | r ∈ R}, G ∪ {π 3 (r) | r ∈ R}, π 4 (R)                        
where the order of the least fixpoint is the pointwise order induced by ⊆ for the first three components and ⊇ for the fourth. Function U * a is a vague analog of a transitive closure, hence the notation. Thanks to it, we can collect all constraints and reduce as much as possible.

To execute a unary operation (an assignment or a guard), we start by running it using the appropriate abstract transfer function. Then we need to build U * a : domains communicate their U n a , compute U a and iterate. In the case of a guard, the condition

SOUNDNESS

265

is the initial constraint given to U * a . In the case of an assignment x := e, the initial constraint is x ← e. Indeed, we recall that the assignment x := e is equivalent to make all variables of x ↓ unknown, and enforcing the constraint x ← e, which is legal since x is (since x ∈ x ↓ ).

Soundness

Soundness is guaranteed because our method relies on reduction: since all reduction steps are individually sound, every combination of them is sound as well. The main problem comes from directed assignments.

We have introduced two kinds of constraints: general guards, and the special case of directed constraints. We recall that the directly constraint x ← e stands for the constraint x = e, but it can only be used when x = , so that reduction only happens in one direction, and thus can be run as an assignment, which is much more efficient than an arbitrary comparison. We shall make sure that no reduction involving x happens from the point where x is made unknown to the moment we run the directed constraint.

In the example detailed in subsection 20.1.1, we see that directed constraints are correctly used by increasing the ghostliness of involved variables. This is the point of item f of assumption 20.10 "Hypotheses on constraint triggering" (page 261).

Theorem 20.1 Let u ∈ U and G := π 3 (U * a (u)) ∈ P (G ). We have a) if u ∈ C , ∀g ∈ G, ∀u ∈ g, u ∈ C b) if u ∈ A , ∀(g 1 , g 2 ) ∈ G 2 , let (N 1 , E 1 , f 1 ) := g 1 in let (N 2 , E 2 , f 2 ) := g 2 in ∀(n 1 , n 2 ) ∈ N 1 × N 2 , (f 1 (n 1 ) ∈ A ∧ f 2 (n 2 ) ∈ A ∧ π 1 (f 1 (n 1 )) = π 1 (f 2 (n 2 ))) ⇒ n 1 = n 2 ∧ (N 1 , E 1 , f 1 ) = (N 2 , E 2 , f 2 ) ∧ π 1 (f 1 (u )) ∈ π 1 (u) ↓
Let us reword this theorem, before justifying it. Let u be a constraint. We name G, the set of constraint graph generated from constraint u. If u is a simple constraint, all constraints generated are simple constraints as well. If u is a directed constraint x ← e, then all directed constraints in G are about variables ghostlier than x. Moreover, a ghost variable can appear at most once as the left-hand side of a directed constraint.

There are some hidden hypotheses, that are parts of the assumptions on generic abstract domains. Relevant ones are:

-each domain allocates and kills only variables that it owns; -triggering of constraints satisfies n (increasing ghostliness); -in a constraint-DAG, a ghost variable can only appear once in the left-hand side of a directed constraint. Let us draw a proof sketch. We made the assumption that real assignments never use their left-hand side in their right-hand side. Thus, the tree of variables that are ghostlier than the left-hand side and the forest of variables ghostlier than variables in 266CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT the right-hand side are disjoint. This has a crucial consequence: all left-hand sides of assignments may never appear in the right-hand side of a directed constraint or in a comparison. Indeed, in the example, the last directed constraint assigns variables under y using variables under low and high.

We now have to check that an assigned variable can only be written once. Let v ∈ V . We distinguish two cases:

-v ∈ V. A directed constraint can only be about a ghost variable (thank to n ). So v is only constrained by the real assignment. So, only once. -v / ∈ V. There is a variable v , a role R and an integer i such that v = R(v ) and v ∈ V i . Since each directed constraint triggered by another directed constraint writes in a variable exactly one level more ghostly, a directed constraint on a variable in V i can only be generated at the i th step of recursion. Moreover, it can only come from the domain that owns the role R. Consequently, there is only one constraint-DAG that may contain a directed constraint on v. As we assumed that a variable may only appear in one directed constraint in a constraint-DAG, there is only one directed constraint to v. So, variables under the left-hand side of a directed constraint appear at most once and only as the left-hand side of a directed constraint, and these variables were previously set to by Assign. Hence, the hypothesis on directed constraints holds.

Termination

A priori, U * a yields infinite sets, which is quite embarrassing to actually compute and to enforce these constraints. But thanks to items h and i of assumption 20.10 "Hypotheses on constraint triggering" (page 261), we can prove that these sets are in fact finite. Indeed, each domain has a well-founded order ⪡. When called, U is given a subset of U (its second parameter), which is the set of constraints the domain is allowed to use. It returns a subset of these constraints, that are lower, according to order ⪡. In the product, to compute U a form individual U n a , we need the intersection of these sets of allowed constraints. So that, at each iteration of U a , to compute U * a , we allow only decreasing sequences of constraints, with respect to a well-founded order (defined as the intersection of well-founded orders).

Here, the hypothesis is local to each domain, making the termination proof distributed. But if the orders are poorly designed, satisfying all of them can be too constraining, and quickly, the set of allowed constraints can be empty, even before being able to perform interesting reductions. In this situation, variables that are ghostly enough will not be reduced. There is a trade-off: either, we prefer to keep termination proof distributed (even if it means that some variables will not be reduced), or we can use the same well-founded order for all domains. This second approach is more predictable but may imply to redesign the order when we add new domains to the product. A hybrid solution can be preferred: trying to stay as close as possible from a reference order, though domains can define their own order to take all their idiosyncrasies into account.

Let us define such an order that is good enough to run the example and that is very close to the order we use in the implementation, up to some technical details. It is a particular instance but that is good enough in practice and thus is worth to be explained in details. Let us examine how the example of subsection 20.1.1 "A Detailed Execution of an Example with Ghost Variables" (page 253) works. We see that ghostlier and ghostlier variables appear in the left-hand side of directed constraints with ghostlier and ghostlier expressions.

Let us take a look to the tree of ghost constraints. From y = low | (high << 16), at the first step of recursion we got -Slice It is clear that the left-hand side of directed constraints decreases, as prescribed by the item f of assumption 20.10 "Hypotheses on constraint triggering" (page 261). Sadly, it is not enough to ensure termination: since we can add ghost variables at each call of U, we may end up into adding an infinitely deep tree of ghost variables. It is also worth noticing that all new ghost variables are under the left-hand side of an assignment, which is pretty natural.

First let us define the complexity order on expressions.

Notation 20.2

For e ∈ E , we denote MVar (e) the multiset of variables in e, that is

MVar : E → M (V ) e →    n i=1 Var (e i ) if e = c /n (e 1 , . . . , e n ) ⦃e⦄ if e ∈ V
As it is a multiset, it takes multiplicity into account. For instance, given a variable x, MVar (x + x) = ⦃x, x⦄. See section A.4 "Multisets" (page 373) for basic notations and definitions about multisets, though we recall some definitions that are crucial and not so common.

Definition 20.7

Let E a set and a partial order on E. We denote DM , and called Dershowitz-Manna order (or multiset order) induced by , the relation on M (E) 268CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT defined by

∀(A, B) ∈ M (E) 2 , A DM B :⇔ ∃(X , Y) ∈ M (E) 2 :      X = ∅ ∧ X ⊆ B ∧ A = (B \ X ) + Y ∧ ∀y ∈ Y, ∃x ∈ X : y x     
We denote DM the reflexive closure of DM .

In other words, to get a smaller multiset, we can remove an occurrence of an element x of B to add as many elements as we want that are lower than x. In [START_REF] Huet | Equations and Rewrite Rules: A Survey[END_REF], there is another, equivalent, definition which is also closer to the interpretation.

Proposition 20.1

Given a poset (E, ), DM is a partial order on M (E).

Proposition 20.2

Given a poset (E, ), if is well-founded, then DM is a well-founded order on M (E).

These propositions are well-known properties of multiset ordering (see [START_REF] Dershowitz | Proving termination with multiset orderings[END_REF]). We denote ⋖ the corresponding strict order. This order compares only the multisets of variables in both operands. We do not care about the structure of expressions. For instance, we have x + y ⋖ x == y ? sin(x) : ln(z) because MVar (x + y) = ⦃x, y⦄, MVar (x == y ? sin(x) ? ln(x+y)) = ⦃x, x, x, y, z⦄ and ⦃x, y⦄ DM ⦃x, x, x, y, z⦄.

But, y + y ⩿ x == y ? sin(x) : ln(z) because y appears twice on the left-hand side, but only once in the right-hand side, even if the left-hand side seems much simpler. This order is useful to talk about ghostlier variables. For instance, in Listing 20.2, since pointers p and q alias each other, the computation q-p is translated into Offset (q)-Offset (p), and we indeed have ⦃Offset (q) , Offset (q)⦄ DM ⦃p, q⦄ since Offset (p) p and Offset (q) q 1 int a [START_REF]1 Dereferencing an arbitrary integer[END_REF];

2 int* p = &a[0]; 3 int* q = &a[1]; 4 int delta = q -p;
Listing 20.2: The difference of two pointers But we are also interested into the structure of expression, hence the next order.

Definition 20.9

Let us denote the order relation on E defined by

a b :⇔ a ⋖ b ∨ (a ⩿ b ∧ a ⫑ b)
where ⫑ is the structural inclusion (see definition 19.1 "Structural inclusion" (page 239)). We denote ≺ the corresponding strict order.

It is the lexicographic order induced by ⩿ and ⫑. Unlike ⩿, it allows keeping the same multi-set of ghost variables while simplifying the expression by cutting some branches. For instance, in Listing 20.3, we need a ghost Slice Let v ∈ V be an abstract state. We call depth of v the integer i denoted depth (v), and such that v ∈ V i This integer is obviously unique.

Likewise, a set of variable V ⊆ V is said to have finite depth if the set

{depth (v) | v ∈ V } 270CHAPTER 20.

GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT

is bounded above.

Lemma 20.1

Let V be a subset of V of finite depth. The restriction to V of is well-founded.

Proof. Given two variables v and w, if v w, then depth (v) > depth (w). Thus, for any decreasing sequence of variables in V , the sequence of depth is increasing. Since is well-founded on set of the form 0, n (with n ∈ N), so is on subsets of V of finite depth.

Lemma 20.2

Let V be a subset of V of finite depth, and E := E V (see notation 19.1 "Expressions" (page 239)). The restriction to E of ⩿ is well-founded.

Proof. Since is well-founded on V , DM is well-founded on M (V ), hence the result.

Lemma 20.3

Let V be a subset of V of finite depth, and E := E V . The restriction to E of is well-founded.

Proof. It is well-founded as a lexicographic order of two well-founded orders.

Definition 20.11 -A terminating relation

Let us denote the smallest relation on

U that satisfies a) ∀(v, e) ∈ A , ∀C /n (f 1 , . . . , f n ) ∈ C , n i=1 Var (f i ) ⊆ Var (e) ↓ ⇒ (v, e) C /n (f 1 , . . . , f n ) b) ∀(C /m (e 1 , . . . , e m ), D /n (f 1 , . . . , f n )) ∈ C 2 , m n ∧ ∃σ ∈ S m : ∀i ∈ 1, n , f i ≺ e σ(i) ⇒ C /m (e 1 , . . . , e m ) D /n (f 1 , . . . , f n )
where S n is the set of permutation of 1, n (see section A.6 "Standard Objects" (page 376)),

c) ∀((v, e), (v , e )) ∈ A 2 , v v ∧ e ≺ e if v is freshly allocated e e otherwise ⇒ (v, e) (v , e )

BINARY OPERATIONS

271

Let us reword that relation in English. Item a explains that we can generate a condition from a directed constraint as long as all variables used are ghostlier. Item b allows triggering a condition from a condition when the new condition involves fewer or as many expressions, and that each of them is lower (with respect to ≺) than an expression of the old condition, which is used at most once. Item c is about generating a directed constraint from a directed constraint. The left-hand side must be strictly ghostlier but the condition on the right-hand side is more complicated. It must be lower with respect to if the variable is newly allocated, otherwise, we need only the right-hand side to be lower or equal.

Theorem 20.2

Let V be a subset of V with finite depth, and U := U V (see definition 19.5 "Elementary ghost constraints" (page 248)). Relation is well-founded on U .

Proof. Previous lemmas directly imply that is well-founded on A V and on A V . Moreover, since we can go at most once from A V to A V , if there is an infinite decreasing sequence of elements of U , then it must either be an infinite decreasing sequence of elements of A V or have an infinite decreasing suffix of elements of A V . By contraposition, is well-founded on U .

This does not directly provide termination, because this theorem assumes that we only work with a subset of V of finite depth. The trick is that order consumes complexity on expressions each time we allocate a new ghost variable (this is item c of definition 20.11). We can only allocate a ghost variable under the left-hand side of a directed constraint, thus, no variable under the right-hand side. Indeed, we recall that we assumed that the left-hand side of assignments never appear in the right-hand side. Thus, we can consume complexity on the right-hand side of directed constraints only finitely many times. The consequence is that we can only allocate a finite depth of variables, from a finite support, thus, the last theorem applies. We just need to take V to contain the actual support and any variables that can be added in the maximum number of allowed allocations. Name M this number, we can only allocate variables that are at most M levels under a living variable: this set has finite depth.

This relation is the one all domains use. Once again, this order is good enough for all cases we met, but it has not particular status. We can use any terminating order, and it is likely we need to adapt in the future.

Binary Operations

An Illustration of the Problem

In our concrete framework, binary operations are only union. In practice, they also include widening, intersection and narrowing (for termination and refinement). But these operations are difficult to perform with operands that have different supports: if 272CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT a ghost variable exists in only one operand, it is not clear what to do with it. We need to know the semantics of the ghost variable to decide, but only the domain that owns is aware of the semantics. Thus, binary operations are split into two steps: we unify the supports, and we run the actual binary operation. Domain unification is the role of function B of a generic abstract domain. Before diving into the details, let us illustrate the solution and the problem on an example. To compute the state after line 6, we need to compute the union of these states, but this is not possible directly. Since the supports are different.

We can see there are 3 regions: -bits 0 to 15 of low are, in both branches, bits 0 to 15 of a ghost variable; -bits 16 to 27 of low are zeroes in a branch, and known to be bits 16 to 27 of a ghost variable in the other branch; -bits 28 to 31 of low are zeroes in both branches. The union is trivial and precise for bits 0 to 15 and bits 28 to 31. But, the domain know that it will have nothing to say for the region between bits 16 and 27. Thus, to unify the support, it is allowed overapproximate the states; hopefully, unification can be done without loss of precision.

After line 3, the state becomes [0:15] (low), which can be remembered, for instance by Equality domain, to be equal to x. Bits 16 to 27 are unknown, and bits 28 to 31 are zeroes. We can now delete Slice [16:27] [16 :27] (low) that is useless. In practice, we can compress steps not to allocate Slice [16:27] [16 :27] (low) at all, since it is easy to predict that it will be useless after the union. The implementation indeed avoids allocating such short-lived variables to save time, but it causes strange intermediate states. In this example, we would not allocate Slice [16:27] [16 :27] (low) in the else branch. Consequently, after unification, bits 16 to 27 of low are . One may think it is a huge loss of precision, but the domain allows it only because it will eventually happen. Yet, it may be surprising to see such not-immediately-necessary overapproximation.

Formal Description

To perform support unification, domains have a function

B : N * × D × D → (P (V ) × P (V ) × G ) 2
The function takes two states whose support should be unified and a positive integer. The integer is the layer of ghost variables we need to unify. For each state, the function return a set of variables to add, a set of variables to kill and a constraint-DAG.

For each level i, B should be called with states where ghost variables of level 0 to i -1 are already unified, and it unifies ghost variables of level i (i.e. variables in V i ). These constraints are formally expressed in assumption 20.11 "Assumptions on support unification function" (page 262).

274CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT

To unify the whole support, we need to successively call B for each level, increasingly. Returned constraint-graphs are evaluated just like for unary operations, and we should collect all constraints recursively to get the best precision.

Ensuring termination of this process is quite tricky. Since each round consists in applying unary operations on the two abstract states, they will all clearly terminate. But, we should still have a finite number of rounds.

Unlike regular assignments that can lead to a finite but arbitrarily high number of new ghost variables depending on the right-hand side expression, unification assignments are meant to make both states similar. The form of the forest of ghost variables can change, but its depth must stay the same. It is a reasonable constraint since adding a variable where there was none means guessing some information from nothing, which seems dubious. Thus, to ensure termination, we dictate that the depth of the ghost variable tree should not increase. The depth of an abstract state is well-defined since the support is always finite. After unification of a and b, the depth of resulting states a and b should not exceed max (depth (a) , depth (b)) + 1. It is not a natural property but a political one. It is enforced by ignoring creation of new variables too deep and constraints about them. Actually, with reasonable domains, especially domains described in following chapters, unification never allocate variables beyond the limit and this forced termination protocol is never triggered.

Chapter 21

Base-offset Pointer Domain P ointer abstraction is often necessary in C, and is implemented in Astrée as explained in [START_REF] Miné | Field-sensitive value analysis of embedded C programs with union types and pointer arithmetics[END_REF][START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF]. Though the old domain is perfectly compatible with the new framework for product of domains with ghost variables, there are reasons that make interesting to adapt it into the framework described in chapter 20 "Generic Abstract Domains and Reduced Product" (page 253).

In this chapter, we will see how this domain work from a mathematical point of view. We will compare the old implementation with what the new framework allows, and describe the design of the new version of this pointer domain, from a quite formal point of view. Implementation aspects will be treated later.

Abstracting Pointers

This section is about the formal foundations of pointer abstraction described in [START_REF] Miné | Field-sensitive value analysis of embedded C programs with union types and pointer arithmetics[END_REF][START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF], and it can be safely skipped if the reader is comfortable enough with this abstract domain.

In most cases, to build pointer values, we take the address of a variable, and perform pointer arithmetic. A pointer can be seen as the address of the beginning of a variable, and an offset relative to this address. Let us denote V the set of variables. Pointers are elements of V × Z. If the offset is too big (or negative), the pointer points after the end of the variable (resp. before the variable). It is not a problem as long as we do not dereference it. Thus, we allow even illegal pointers. For instance, in Listing 21.1 (where we assumed sizeof(int) == 4), after lines 3 and 4, the pointer is out of x, dereferencing it would be an error, but the pointer can still be modified and made valid before being dereferenced.

An imprecise pointer value is a set of such ordered pairs, that is an element of P (V × Z). We abstract it as an ordered pair of P (V) × P (Z). The first component is bounded by the cardinal of V, and can be represented explicitly. The second component is delegated to be abstracted by a numerical domain.

Pointer arithmetic is translated into operations on the offset. Adding or subtracting an integer directly performs the same computation on the offset, up to a constant factor Listing 21.1: A pointer that is valid, then out of bounds, then valid again depending on the type. To compute the distance between two pointers, we need the two pointers to point to the same variable, and we compute the difference of offsets, divided by the size of the type. To check that two pointers points to the same variable, sets of pointed variables should be equal singletons.

We can represent well-formed pointers with this technique, but we need additional special values for non-regular cases. Especially, we add a special value NULL, for C's null pointer, and a special value INVALID for all values that have not been obtained from the address of a variable and some pointer arithmetic. For instance, pointers obtained with bitwise operations, like explained in previous chapters, will result in special value INVALID. We can hope to refine such a pointer into a precise pointer, using other domains.

Old Implementation vs. New Framework

As explained in subsection 13.2.3 "The Stack of Domains" (page 167), and especially shown on Figure 13.1 "Structure of the composite domain of Astrée " (page 168), the current pointer domain is parametric: it takes a numerical domain as parameter that it uses to represent offsets and numerical values.

In the old implementation, variables are represented by identifiers chosen by the struct domain. Pointer domain uses the same identifier for the numerical values in its underlying domain, and thus can use only one numerical value for each variable. This has a serious consequence on the representation of variables that may hold an integer and pointer value. To know if the underlying numerical value should be interpreted as an offset or directly as the value of the variable, there is an additional special base. While pointer values are represented by a variable the pointer points to and an offset, for numerical values, this base variable is a special value NUM. An integer value n will be represented by the pair (NUM, n). In a case where integer and pointer values are mixed, this leads to major imprecision. Let us illustrate on Listing 21.2. In the if-branch, p get a numerical value 42. The representation is (NUM, 42). In the else-branch p points to variable x, with a byte-offset of 4; the representation is (x, 4). After line 7, once the union is performed, the abstract value is ({NUM, x}, {4, 42}). At this point, the abstract value is very imprecise: the concretization is made of numerical values 4 and 42, and pointers to x with offset 4 and 42. Since pointers can have arbitrary numerical value, and numerical values can point anywhere, this abstract value is supremely imprecise. The new framework allows a much smarter representation. Numeric and pointer domains are at the same level, and share the same variables. Thus, the numerical domain can store directly the value of each variable. Pointer domain may allocate a ghost variable to store exclusively the offset. We can go even further and allocate multiple ghost variables to be the offset of each base variable, including for the NUM special base. This allows possible numerical values and pointer values not to be mixed. For instance, in the example of Listing 21.2. We would obtain the abstract value {(NUM, {42}), (x, {4})}. The effective value of p is still unknown, but we do not mix both values. Thus, if we test whether p != 42, then, we know that p points to x [1].

The new framework allows several domains to represent pointers and to cooperate. In the old version of the pointer domain, the variable y of the example of Listing 20.1 "A simple example that needs ghost variables" (page 253) would receive the value INVALID as it is not constructed by taking the address of a variable and performing pointer arithmetic. Having a sibling domain that is more comfortable with this kind of operations can help to refine the state, by sharing the constraint x = y, which allows the pointer domain to refine INVALID into the value (t, idx).

Indeed, some special cases of bitwise operations are implemented in the pointer domain, with some assumptions on alignment. This used to be the only solution, but it is quite immoral from a software engineering point of view. In the new framework, it is easy to add a sibling domain that is able to handle another kind of properties.

Another benefit of the adaptation of the pointer domain is that it can use the top level numerical domain rather than having its own underlying one. Using a single instance of the numerical domain allows it to express relation between real numerical values and offsets.

Adaptation in the New Framework

Let us give some formal aspects of the adaptation of the pointer domain in the new framework. Abstract transfer functions are quite similar to the old version, we will not linger on that matter. We are mainly interested into ghost variable-related elements.

First, this domain defines a single role: Offset. If we want an offset for each base, we would need a role Offset v for each variable v. At first, we use only one role Offset to be as close as possible to the old implementation.

Generated constraints are simply translations on the offset of the effects of pointer arithmetic. For instance, the instruction q = p + i, where p is a pointer and i and integer, generates the directed constraint Offset (q) ← Offset (p) + i × sizeof(t) where t is the type pointed by p. int x [START_REF]1 Dereferencing an arbitrary integer[END_REF]:

int* p = &x[1]; int* q = &x[3]; int diff = q -p;
Listing 21.3: Difference of two aliasing pointers Though most reductions are quite straightforward, there is a quite natural case where the intuitive approach fails. Let us examine Listing 21.3. After line 3, p and q point to x, with offsets Offset (p) and Offset (q), which are respectively equal to 4 and 12 (assuming sizeof(int) == 4). The assignment of line 4 is correct since both p and q point to x. It is appealing to generate from this assignment the directed constraint diff ← Offset(q)-Offset(p)

4

, but this would be illegal, since it does not comply the rule of increasing ghostliness (see item f of assumption 20.10 "Hypotheses on constraint triggering" (page 261)). Indeed, ghost variables have an owner, which is the only domain that can generate directed constraints about it (and thus, assign it). Here, diff is a real variable, thus it is not owned: all domain will perform the real assignment, and thus diff is not necessarily , and thus, the reduction is not equivalent to an assignment. There are two workarounds, but both are based on the same idea: not to use a directed constraint. The (undirected) constraint diff = Offset(q)-Offset(p) 4 is correct and respects all hypotheses. The two possibilities are either to issue it in ghost variables reduction process, or delay it until the ordinary reduction, à la [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF].

Due to its simple usage of ghost variables, support unification prior to binary operations is also pretty elementary. The only possible case where there is something to do is when a variable in an operand has an offset while the same variable in the other operand does not. The simplest strategy is to add the missing offset variable with a ⊥ value. Since this variable is unused, the concretization of the whole state is not empty. This might be a problem with domains implementing ⊥-coalescence that have been implemented in the old framework and is not aware that there are ghost variables, and ⊥-coalescence does not apply if the ghost variable is unused. Fortunately, it is not necessary to initialize the new unused ghost variable with ⊥; any overapproximation is correct. Providing an arbitrary value can lead to loss of precision, but we can initialize the new ghost variable to the value it has in the other operand, where it is defined. This domain complies with the termination relation defined in section 20.4 "Termination" (page 266). Let us explain the intuition. The only relevant things this domain can deduce are about offsets. Translating a pointer operation in the offset world increase the ghostliness of constraints. There is still the problem of allocation of new ghost variables, that could make the depth of tree of ghost variables infinitely deep. This domain allocates a ghost variable only when a variable, which is newly allocated or used to hold a numerical value, gets a pointer value. In this case, the offset of the left-hand side receives the offset of the right-hand side, which is strictly ghostlier, ensuring termination.

Possible Improvements

Since a perfect analyzer does not exist, we should keep in mind that our objective is to be able to perform analysis without false alarms on software we are interested in, and make it fast enough not to exhaust user's patience. From that perspective, and given the current applications, this domain does not need improvements. Yet, we can envisage future applications that could highlight limitations of the current state of this domain. Let first note that the new implementation of this domain is more precise than the old one * . Every precision problem explained thereafter also applies to the old implementation, though the proposed solutions for the new implementation might not be feasible in the old one.

The most radical precision improvement, as mentioned before would be to define roles Offset v for all variable v. Offsets of different bases would not be mixed anymore. This would probably have a major impact on performance as we need to apply ghost constraints on each ghost variable, which can be many. A hybrid solution would be to keep the role Offset, and only use Offset v for variables v for which a distinct offset could increase the precision. This can either be decided by heuristics or provided as an analysis hint by the user. Indeed, in most cases, when a pointer can point to several variables, they are analogous, and the respective offsets are too.

There might still be interesting to add a role to define ghost variables that store the pure numerical value. For now, this is simply the value of the variable. But when a variable can be a pointer, its numerical value can be arbitrary, and thus we lose any purely numerical data. By adding a role N umericValue † , one could isolate the numerical values aside. The integer interpretation of the variable would still be imprecise, but we would know that it can be a precise integer value, or a precise pointer value. This would be useful for programs where variables can be used with two different types. This is idiomatic in Python and other dynamically typed languages: since we can test the type of variables at run-time (e.g. using isinstance), we can distinguish cases. In statically typed languages, like C, there is no way to know if a variable contains a pointer value or an integer, thus it is not that common to mix both kinds of values. It can still happen as a tagged union, but then it would be relevant to partition according to the tag.

Chapter 22

Slices Domain

T he main application that motivated the new framework is the precise abstraction of bitwise operations. This chapter details this domain. We will show our choice of roles, explain it and exhibit their semantics. Then we will describe the abstract properties handled by this domain, and the concretization. Finally, we will explain how to perform ghost reduction.

In the following, we will assume this domain work with a simple equality domain. We need it to decide whether two variables are definitely equal or not. It is required only to implement two cases: it has to deduce that two variables x and y are equal when it gets the assignment y := x or the constraint x = y (of course including the directed constraint x ← y). Without this very elementary domain, slices domain is much less interesting.

For the sake of simplicity, we will assume that variables are all 32-bit wide, and consequently the set of values I is 0, 2 32 -1 .

Roles and Semantics of Ghost Variables

We already saw roles of this domains in examples. Formally, this domain defines roles Slice

(v) [a : a + s] = v [b : b + s]       
We see that we only allow the domain to express equality between contiguous sets of bits. This implies that the statement y = x & 0xff00ff requires the creation of two ghost variables: Slice [16 :23] (y) that both get the value of x. More complex roles could allow stating the equality of arbitrary patterns, but this solution was not retained. Indeed, it is much more complex to handle in the implementation: contiguous slices are naturally orderable, but arbitrary patterns are not. Especially, it is much easier to check that parts of a sliced variable are disjoint when we only use contiguous slices.

Moreover, extracting non-contiguous bits is not a common operation in our context. Building segment descriptors (see subsubsubsection 2.3.2.2.1 "Segment Descriptor" (page 19)) or paging structures (see subsection 2.3.3 "Paging" (page 25)) only uses contiguous slices.

From a more theoretical point of view, any set of role that is able to express equality of two arbitrary bits of two variables is expressive enough. If the number of roles is an issue, we can reduce it to 32 2 = 1024, we would only need the roles Informally, the semantics of Bit a b (v) is the set of states where the a th bit of Bit a b (v) is equal to the b th bit of v. Reducing the number of roles can be useful for languages where types have a run-time cost (assuming we cannot share a single type for all roles of the same form). The drawback is obvious: this setup requires to allocate many ghost variables. Since ghost variable allocation is relatively slow, using slices seems to be a good trade-off between performance and maintainability.

Abstract Values and Concretization

This domain establishes relations between ghost variables it owns and their immediate parents, but from the point of view of each layer of variables, the abstraction is nonrelational. In particular, there is no relation between ghost variables under distinct real variables. Variables are split in non-overlapping segments of bits. Each segment can have 4 states: zeroes, ones, unknown or a slice of an immediately ghostlier variable.

Formally, we define the set of values of a slice * :

H := {ZERO, ONE, } (a, b) ∈ 0, 31 2 0 b -a 30
The meaning of ZERO, ONE and are quite intuitive. Pairs of integer stands for a slice of a ghostlier variable. We do not need to name it since its name is deducible from the bounds of the represented slice (slice of the parent variable) and the representing slice (slice of the ghost variable). The set of slices is defined as † :

Y := (i, j, s) ∈ 0, 31 × 0, 31 × H 0 j -i s ∈ 0, 31 2 ⇒ j -i = π 2 (s) -π 1 (s)
Slices are made of 3 components: the first index, the last index and the content of the slice. When the content of the slice is a slice of a ghostlier variable, sizes must be the same, and such a slice can be at most 31-bit long.

The state of a variable is a sequence of slices where indices form a partition of 0, 31 , and is formally defined as ‡ :

p :=          (s i ) i∈ 1,n ∈ Y n 1 π 1 (s 1 ) = 0 π 2 (s n ) = 31 ∀i ∈ 1, n -1 , π 2 (s i ) = π 1 (s i+1 ) + 1         
Expression Y stands for the Kleene star of Y (see section A.6 "Standard Objects" (page 376)).

An abstract state is a map from a support V to p, with some constraints. If a variable v has a slice (i, j, (a, b)), the variable Slice

D :=                  f : V p ∀v ∈ supp (f ) , let (s i ) i∈ 1,n := f (v) in ∀i ∈ 1, n , let (c, d, (a, b)) := s i in π 3 (s i ) ∈ 0, 31 2 ⇒ Slice [a:b] [c:d] (v) ∈ supp (f )                 
In practice, we also want ghost variables belonging to the slice domain to be made of a unique slice. Indeed, given a variable x and a ghost variable Slice

γ : D → D f →                      g : supp (f ) → I ∀v ∈ supp (f ) , let ((a i , b i , s i )) i∈ 1,n := f (v) in ∀i ∈ 1, n ,      s i ∈ 0, 31 2 ⇒ g ∈ ❲ Slice [π 1 (s i ):π 2 (s i )] [a i :b i ] (v) ❳ s i = ONE ⇒ v [a i : b i ] = 2 b i -a i +1 -1 s i = ZERO ⇒ v [a i : b i ] = 0                     
In other words, if a slice is equal to a slice of a ghost variable, this relation must be true in the concretization. If a slice is ONE, then all bits of this slice are ones. If a slice is ZERO, then all bits of this slice are zeroes. If it is , the slice can have arbitrary value: there is no constraint. Using the semantics of ghost variables, the concretization is quite direct.

Ghost Constraints

As usual, there are two kinds of reductions:

-reductions that are useful to keep the abstract state precise, constraining only ghost variables belonging to the slices domain, which are mainly expressed as directed constraints; -reductions that communicates interesting facts to other domains about variables that do not belong to the slices domain, which are actually what the domain is made for, e.g. notifying other domains when a value is rebuilt through bitwise operations. We will detail these categories successively.

Directed Constraints

Directed constrains are generated to react to a directed constraint or to an assignment. Until now, expressions where arbitrary trees of constructors, but we need to open the structure of expressions to explain the reductions.

There are two categories of operators: bitwise operations that are precisely abstracted, and other operations (including arithmetic ones) that are handled very coarsely. Result of operations of the second kind are abstracted by . Operations that are precisely abstracted are divided in three categories:

-unary operators: bitwise negation (~); -binary operators between two sliced values: bitwise and (&), bitwise or (|), bitwise xor (^), join ( ) and meet ( ); -binary operators whose right operator is a literal integer: left shift (<<), and right shift (>>).

It is to emphasize that a shift with a non-literal second operand is considered as an computation that the domain cannot abstract precisely.

For operations of the second category (binary operations with two sliced operands), variables must be unified, that is we should transform them so that they share the same partition of their bits, though content can be different.

We will show how to evaluate a directed constraint (or an assignment) for a single slice (i, j). These rules should be applied on each slice, independently. Another step may follow to perform compression. Slices may not been known in advanced and simply cut again as needed.

Given a slice fine enough and an abstract state, we define the function

⦑_⦒ : E → {ZERO, ONE, } ∪ ( 0, 31 × 0, 31 × E )
that gives the value of a single slice as one of the three constants ZERO, ONE and , or a 3-tuple (a, b, e) meaning that this slice is the equal to the slice a to b of expression e. where ≡ means that these expressions are provably equal. We can see that, for A, we need to decide the equality of ghost expressions, and thus, we need the equality domain mentioned in the introduction of the chapter. The slices domain can ask the equality domain if two variables are equal § using the input communication channel (see section 12.3 "Partially Reduced product" (page 159)) about the precondition.

Likewise, we can give the rules for other binary operations. § Of course, this is a sound approximation: two variables may be in fact equal, while the equality domain does not know it. Indeed, it has limited deduction power, this is why we design abstract domains (like the slices domain) to deduce more equality properties. where

A :=      (a, b, u) if (a, b) = (c, d) and u ≡ v ⊥ otherwise
Shifts do not modify the content of slices, but change their position in the result. Before stating the semantics of shifts, we will define functions that keep only bits 0 to 31 of modified sliced expressions, that is sliced variables that are not subjected to the usual bounds 0, 31 . Let us define modified sliced expressions. First, the state of a single slice, then a slice, the set of sliced expressions and modified sliced expressions

H := {ZERO, ONE, } {(a, b, u) ∈ 0, 31 × 0, 31 × E | 0 b -a 30} Y :=      (i, j, s) ∈ 0, 31 × 0, 31 × H 0 j -i s ∈ 0, 31 × 0, 31 × E ⇒ j -i = π 2 (s) -π 1 (s)      p :=          (s i ) i∈ 1,n ∈ Y n 1 π 1 (s 1 ) = 0 π 2 (s n ) = 31 ∀i ∈ 1, n -1 , π 2 (s i ) = π 1 (s i+1 ) + 1          p :=          (s i ) i∈ 1,n ∈ Y n 1 π 1 (s 1 ) 0 π 2 (s n ) 31 ∀i ∈ 1, n -1 , π 2 (s i ) = π 1 (s i+1 ) + 1         
Modified sliced expressions are sliced expressions but which are can cover more than 32 bits. Their primary purpose is to be an intermediary in shift operations, before cutting the surplus. More precisely:

-expression-slices (H ) are intermediary values in computations, slices refer to an expression (and not a ghost variable); -positioned expression-slices use expression-slices; -sliced expression of p use expression-slices, but respect the 0, 31 bounds; -modified sliced expressions of p are like expressions of p , but they may overflow the bounds 0, 31 . They will be cut later to get a value of p . Now, let us define the cut-left function, that removes everything below the bit 0:

⧘_ | : p → p ((a i , b i , s i )) i∈ 1,n → a i , b i , s i i∈ l,n
where

                     l ∈ 1, n 0 ∈ a l , b l ∀i ∈ l + 1, n , (a i , b i , s i ) = (a i , b i , s i ) (a l , b l ) = (0, b l ) s l ∈ 0, 31 2 ⇒ s l = (π 1 (s l ) -a l , π 2 (s l ))
s l ∈ 0, 31 2 ⇒ s l = s l the cut-right function, that cuts everything beyond bit 31:

|_ ⧙ : p → p ((a i , b i , s i )) i∈ 1,n → a i , b i , s i i∈ 1,h
where

                     h ∈ 1, n 31 ∈ a h , b h ∀i ∈ 1, h -1 , (a i , b i , s i ) = (a i , b i , s i ) (a h , b h ) = (31, b l ) s h ∈ 0, 31 2 ⇒ s h = (π 1 (s h ) , π 2 (s h ) -b h + 31) s h ∈ 0, 31 2 ⇒ s h = s h
and the composition, that keeps only slices for existing bits (i.e. bits from 0 to 31):

⧘_⧙ : p → p x → ⧘ |x ⧙ |
We can now define the result of shifts. Let ((a i , b i , s i )) i∈ 1,n ∈ p be the sliced expression of variable e and p be an integer. The result of e << p is

⧘ a i , b i , s i i∈ 0,n ⧙ where ∀i ∈ 1, n , (a i , b i ) = (a i + p, b i + p) (a 0 , b 0 ) = (0, p -1) s 0 = ZERO Similarly, the result of e >> p is ⧘ a i , b i , s i i∈ 1,n+1 ⧙ where ∀i ∈ 1, n , (a i , b i ) = (a i -p, b i -p) (a n+1 , b n+1 ) = (32 -p, 31) s n+1 = ZERO
For an instruction e of any other kind, including a variable, we have ⦑e⦒ = (0, 31, e)

The second step is compression. We define the function

⦕_⦖ : p → p ((a i , b i , s i )) i∈ 1,n → a i , b i , s i i∈ 1,m such that ∃f : 1, n → 1, m : f (1) = 1 ∧ f (n) = m ∧ ∀i ∈ 1, n -1 , f (i + 1) ∈ {f (i), f (i) + 1} ∧ ∀i ∈ 1, n -1 , s i ∈ {ZERO, ONE, } ∧ s i = s i+1 ⇒ f (i) = f (i + 1) ∧ ∀i ∈ 1, n -1 , s i ∈ {ZERO, ONE, } ∧ s i = s i+1 ⇒ f (i) + 1 = f (i + 1) ∧ ∀i ∈ 1, n -1 , (s i , s i+1 ) ∈ ( 0, 31 × 0, 31 × E ) 2 ⇒ let ((α 1 , β 1 , η 1 ), (α 2 , β 2 , η 2 )) := (s i , s i+1 ) in β 1 + 1 = α 2 ∧ η 1 ≡ η 2 ⇒ f (i) = f (i + 1) ∧ ∀i ∈ 1, n -1 , (s i , s i+1 ) ∈ ( 0, 31 × 0, 31 × E ) 2 ⇒ let ((α 1 , β 1 , η 1 ), (α 2 , β 2 , η 2 )) := (s i , s i+1 ) in β 1 + 1 = α 2 ∨ η 1 ≡ η 2 ⇒ f (i) + 1 = f (i + 1) ∧ ∀i ∈ 1, n -1 , (s i , s i+1 ) ∈ ( 0, 31 × 0, 31 × E ) × {ZERO, ONE, }, f (i) + 1 = f (i + 1) ∧ ∀i ∈ 1, n -1 , (s i , s i+1 ) ∈ {ZERO, ONE, } × ( 0, 31 × 0, 31 × E ) , f (i) + 1 = f (i + 1) ∧ ∀i ∈ 1, m , a i = a min {j∈ 1,n | f (j)=i} ∧ ∀i ∈ 1, m , b i = b max {j∈ 1,n | f (j)=i} ∧ ∀i ∈ 1, m , let j := min {j ∈ 1, n | f (j) = i} in let j := max {j ∈ 1, n | f (j) = i} in s j ∈ {ZERO, ONE, } ⇒ s i = s j s j ∈ ( 0, 31 × 0, 31 × E ) ⇒ s i = (π 1 (s j ) , π 2 s j , π 3 (s j ))
This function is well-defined since f is completely constraint: we know that f (1) = 1, and step by step, we can build it entirely. In natural language, ⦕_⦖ merges consecutive slices whose values are contiguous. For instance, the expression

(x & 0xf) | (x & 0xf0) | 0xf00 | 0xf000 | (0xf0000 & 0) | (0xf00000 & 0)
gives the uncompressed sliced expression s := (0, 3, (0, 3, x)), (4, 7, (4, 7, x)), (8,11 Once we got a compressed sliced expression, we can generate directed constraints. Let ((a i , b i , s i )) i∈ 1,n be a sliced expression that is assigned to a variable x. There are two possibilities: the sliced expression may be trivial are not. It is trivial if it is made of a single slice. For instance (k | 0xffff) | (k | 0xffff0000) gives the sliced expression (0, 31, ONE), which is not trivial since it is not the original expression. But, (k & 0xffff) | (k & 0xffff0000) gives the sliced expression (0, 31, (0, 31, k)), which is trivial.

If the sliced expression is not trivial, we can emit interesting directed constraints. For all i ∈ 1, n , if s i has the form (α i , β i , e i ), we shall generate the directed constraint

Slice [α i :β i ] [a i :b i ] (x) ← e i after having allocated Slice [α i :β i ] [a i :b i ] (x) if needed.
Unused ghost variables under x can be removed.

Other Reductions

If the sliced expression is trivial, we can again distinguish two cases. Either the slice is the original expression or not. We have given two examples of the latter case above. Let us give an example of the former: the expression (k & 0xf) + l simply gives the sliced expression (0, 31, (0, 31, (k & 0xf) + l)). In this case, the domain deduced nothing interesting. When the expression e result in the trivial sliced expression (0, 31, (0, 31, f )) with e = f the sliced expression is not uninteresting, and thus we should emit a constraint. But the constraint x ← f , when x is the assigned variable, does not match the requirements of directed constraints. But we can generate the regular constraint x = f , which can be helpful. In particular if the f is a single variable, when the equality domain receives this constraint, it merges equivalence classes of x and f . This is what happens when a variable is reconstructed bitwise, like in the fourth line of Listing 20.1 "A simple example that needs ghost variables" (page 253).

This reduction is the primary purpose of this domain, but it can emit other reductions. For instance, if, in the assignment x = e, e results in the sliced expression ((0, i, (0, i, f )), (i + 1, 31, ZERO)), with i ∈ 0, 30 , the domain can deduce the constraint x ≡ f 2 i+1 . This may be useful for a numerical domain.

Possible Improvements

A few improvements have been considered. The first one is to generalize the pattern of ghost variables, to allow more than contiguous slices. For now, a bit-segment of a ghost variable can be used to represent a bit-segment of its parent variable. We may want to generalize this to any pattern. This matter has been discussed previously and was judged difficult to implement and useless, as we only cut contiguous slices in our application. In the future, for other use cases, it might be useful to represent more complicated patterns using a single ghost variable, if performance is an issue. It would require 2 32 -2 roles (since we exclude a variable that represent no bit, or all bits).

A more complicated but interesting question is the sharing of ghost variables. Ghost variables are useful to keep copies of expressions at a point of the execution. For instance, ghost variables in Listing 20.1 "A simple example that needs ghost variables" (page 253) all store the same value: the value of x (which is not modified). It could be a major improvement to allow sharing ghost variables with a copy-on-write strategy. We could go even further: when a ghost variable is a simple copy of a variable, we could just use the ghost variable, still with a copy-on-write strategy. When a variable is used instead of a ghost variable, it is watched for modifications. If it is modified (or killed), we allocate a ghost variable to store its former value, and we use it instead. Unfortunately, it works only when the expression cut by bitwise operations is a simple variable. Moreover, in most cases, this variable will indeed be killed, thus, we probably should allocate at least one ghost variable in the first place. Sharing ghost variables is only an implementation issue.

One can also add new reductions according to the needs. For instance, any variable whose lowest bit is in a ZERO slice is even; and if the lowest bit is in a ONE slice, the variable is odd. There are many such cases that must be discovered and implemented when they become necessary for future use cases.

Chapter 23

Other Domains

O ther domains in this new framework are interesting enough to be mentioned, but each of them does not deserve a separate chapter. So, they will all share the same. We will first look at adapting existing numerical domains. We will see an example of numerical domain that may use ghost variables, and a domain to handle linear combinations of pointers, that is not implemented for now, as it needs an additional feature in the framework of domain cooperation with ghost variables to be efficient.

Plain Old Numeric Domain

Any domain that existed in the old framework can be adapted for the new one. There are two cases: either it uses ghost variables, and we need to adapt this usage (see chapter 21 "Base-offset Pointer Domain" (page 275)), or it is a simple domain without ghost variables. We want here to explain the latter as it is the case of the composite numerical domain of Astrée. More generally, any domain that does not use ghost variables can be adapted in the same way.

Intuitively, such a domain needs no ghost variable and thus has no role. Ghost variable-related features are trivial, and do nothing. -⪡⊆ U × U : this relation is useful to ensure termination, and should be wellfounded. When domains generate ghost constraints, new constraints must match the relation defined by all involved domain. Here, a domain without ghost variable has no such requirement, and it would be a shame to choose an order that uselessly restrict the set of allowed constraints. There are two ways to avoid this problem:

-If all domains share a common order, as explained in section 20.

U : U × P (U ) × D → P (V ) × P (V ) × G × P (U ) (u, U, a) → (∅, ∅, g 0 , L)
where g 0 is the constraint-DAG that contains no constraint, and L is a set of constraints allowed of the next step, according to the choice made for ⪡. Formally, g 0 := ({n 0 , n 1 }, {(n 0 , n 1 )}, u 0 )

where n 0 and n 1 are two arbitrary nodes and u 0 is a function with empty support. Formally u 0 := ({n 0 , n 1 }, U , ∅)

according to the definition of functions (see section A.5 "Function Theory" (page 374)). This DAG has two nodes, one edge, with no constraint; its source is n 0 and its sink is n 1 . Due to the definition of constraint-DAG, we need at least one edge, thus, we cannot have a DAG made of a single node that is both the source and the sink. This is to reject more easily disconnected constraint-graphs. This remark and such trivial graphs have already been mentioned in section 19.4 "Ghost Constraints" (page 247). This U generates no new constraint, does not allocate or kill new variables. -

B : N * × D × D → (P (V ) × P (V ) × G ) 2 (i, a, b) → ((∅, ∅, g 0 ), (∅, ∅, g 0 ))

COMPUTE THROUGH OVERFLOW

Like U, since the domain defines no role, there is nothing to do for ghost variables. Here, we can see that supports are always unified since the set of ghost variables belonging to the domain is always empty.

Compute Through Overflow

Though existing numerical domains can be adapted to the new framework by making it ignores ghost variables, the converse is not true: there are numerical domains that can be interested in ghost variables.

An example of such a domain is a domain that handles compute through overflow (CTO). To work properly, it needs both signed and unsigned versions of considered variables. But often, only one of them is a real variable, the other is a ghost variable * . We will not define precisely this domain as it is outside the scope of our work, but we will give some aspects related to ghost variables.

This domain needs two roles: SignedOfUnsigned and UnsignedOfSigned. In the old framework, ghost variables where simply variables in an underlying domain, just as the base-offset domain. To adapt a CTO domain to the new framework, directives given to the underlying domain must be concerted into constraints. This is very similar to the adaptation of the base-offset domain (see chapter 21 "Base-offset Pointer Domain" (page 275)).

Though this adaptation is less systematic as the one for domain without ghost variable (see section 23.1 "Plain Old Numeric Domain" (page 293)), the method used for base-offset is a very didactic example whose pattern can be followed with few adaptations.

Linear Combinations

In subsection 18.1.3 "Linear Combinations of Pointers" (page 234), we have explained how linear combinations of pointers may be useful. This could be solved using linear combinations of ghost variables, that store the values of terms of the linear combination. This domain is not implemented for now as it is not useful yet, but it has a major difficulty. Let us show an example to identify it.

Let us study Listing 23.1. In lines 6 and 9, we assign r a pointer to array x, with offset 1 and 3 respectively, to which one adds an arbitrary value p (or the equal value q). At line 11, p is removed from r, so that y is only &x[1] or &x [3]. At line 6, r is the symbolic sum of two ghost variables, containing values of &x[1] and p. These ghost variables can be written T erm 1 (r) and T erm 2 (r). Indices are arbitrary, since there is no meaningful way to discriminate both variables terms: they have the same coefficient, addition is commutative. . . . For the sake of the example, we simply follow the syntactic order. At line 9, we have the same support: two ghost variables under r T erm 1 (r) and 298 CHAPTER 24. IMPLEMENTATION lator" (page 301)). Everything in red has been implemented for the new framework. The struct domain is paced here for reference, so that it is easy to place this composite domain in the global structure of Astrée (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)). We can also remark that we reused numerical domains thanks to an adapter.

There are two important facts to know to understand the implementation. First, the evaluation strategy is very different from the one explained in the formalization. Here, it is a dialog between domains and the driver that manages the execution of ghost constraints: they are still generated and enforced in a distributed way, but constraints all go through the driver that acts as the conductor. Moreover, theoretical aspects are mainly absent from the implementation. Typically ⪡, the relation enforcing the termination as seen in definition 20.3 "Generic abstract domain" (page 258), has no place in the code. It should be satisfied, but the order is not implemented by itself. Likewise, roles are totally implicit: constraints linked to roles should still hold, but they are not made explicit as the variable identifier. There are roles in the implementation, but they are a different notion from the mathematical one, though they often overlap.

Roles

In Astrée, under struct domain, variables represent memory cells and are identified by a tag and an integer. Such a pair is called a "key". Tags encode some information about the type of the variable, but it is a very technical concern we are not interesting in. In the following, the key will be reduced to the single integer field. Keys are generated by the struct domain, as a function of the type, the memory block and the position of the cell in its memory block. To allocate new variables, we need new identifiers, but struct domain can use any integer, thus, we need to dynamically build a correspondence between outer keys, allocated by struct domain, and inner keys, allocated by the driver, that include keys of ghost variables. Everything coming from the struct domain should be translated to use inner keys. Likewise, when something is communicated back to the struct domain, concrete types should be translated to use outer keys. Quite often, values given to the struct domain are not really used, but only provided back to the pointer domain at the next instruction ‡ . To avoid two useless translations, we made an intensive use of abstract types, so that the struct domain cannot introspect these values, and thus they do not need to use outer keys. The old framework was also adapted in the same way to keep compatibility. Indeed, the new and the old framework share the same signature and are still both usable § . In fact, when we do not need the slices domain (or any new domain), we should rather use the old framework since it provides better execution time (see section 25.3 "Performance Evaluation" (page 320)), and it is probably more reliable, because it was already heavily used. Inner keys are generated by the key dispenser. In OCaml, it is usual to choose functional style. Each module defines an immutable type t, that stores the state of the module. Such types usually include fields for underlying modules' t types. Transfer functions does not modify the value in place, but return a new object for the result. Since data are persistent, sharing of substructure allows this style to be efficient.

But for the key dispenser, we need a mutable (imperative) state. This is closely linked to the notion of roles from the implementation point of view. Let us explain what happen when analyzing Listing 24.1. At line 4, we need the creation of a ghost variable to represent the offset of p; at line 7, we also need such a variable (in the else-branch, it was not already allocated). At the end of the conditional branching, we shall join both states. The offset of p of each branch was represented by a separate value. In particular, the state of the else-branch is not aware that a ghost variable allocation occurred in the if-branch. But we cannot use twice the same key for different variable, as the join would merge variables that have nothing in common, thus the state of key dispenser must be shared globally. Moreover, we would like to use the same key for the offset in both branches, so that, after line 8, no renaming is needed, and we can perform the join directly.

For this reason, the key dispenser has a mutable state that is shared between all This way of defining roles enforce a useful property: domain cannot see roles that they do not define. For instance, in Figure 24.1 "New structure of the composite pointer domain of Astrée" (page 298), the combinator on the top define their roles as the disjunction of roles of the bottom combinator and roles defined by slice domain. The bottom combinator builds its roles from roles of the adapter and the base-offset domain. But the slice domain cannot see roles defined by base-offset domain. This way, domains are allowed to allocate only ghost variables they own, since they (usually) need to provide the role of the variable (and its overlaying variable). This allows building the owning relation, which can be checked when ghost constraints are generated or when the ghost variable is killed.

The driver also defines and can own ghost variables. This will be explained later.

Translator

Translating directives and constraints between inner and outer keys is the job of the translator. This module should keep correspondence between inner and outer keys of real variable. It is also in charge of the allocation of new keys. There are two mechanisms.

-For real variables, allocation is ordered by a directive emanating from the struct domain. When the translator is asked to translate an allocation directive for a variable with outer key k, it gets a new inner key from the key dispenser (for the full role Real k) and adds it to the correspondence. -For ghost variables, new keys are requested by domains. To do so, they are given a proxy to the translator, which gets a new key from the key dispenser. The new key is added to the correspondence later, when ghost variables are allocated (just before enforcing ghost constraints).

CHAPTER 24. IMPLEMENTATION

Key translator does more than requesting new identifiers and keeping the correspondence between real variables' inner and outer keys. It also keeps track of the number of uses of ghost variables. Since abstract states can be very complicated, it might be difficult or expansive to check is a variable is still used somewhere. The translator provides a reference counting feature. Domains declare how many times it uses each variable, or the variation in the number of uses. When the number of uses comes down to 0, the variable is marked to be deleted at the next round of ghost variable deletion. By then, it is still alive, but unused.

The translator is owned by the driver (which owns pretty much everything, more or less directly) because it needs to know the inductive definition of roles. Consequently, its state is opaque to domains since they are not aware of roles except their own. But domains need to operate on the translator to allocate new variable, or change the number of uses of a variable (including deleting it). They may also need to access data, for instance, the number of uses of a ghost variable or its owner.

To solve this issue, domains are provided a view of the state of the translator with some modifications, and returns new modifications to apply, like a ghost variable allocation or reference counter update. Each domain receives the same view, but the set of updates is the union of all updates generated by domain executed before. When accessing the data through the view, one should take care to also take updates into account, since at this point, translator's state is unchanged. When the domains return the control to the driver, modifications are applied on the translator.

Of course, utility functions are provided to manipulate view and updates transparently, without having to dive into the concrete structure.

In detail, the view is a record (a product type) containing function values. Let us enumerate the 3 types of modifications with their simplified representation:

-a ghost variable creation NewKey(key, role, uses); -an assignment to the number of uses of a ghost variable SetUse(key, uses); -an offset in the number of uses of ghost variables AddUse(key, delta_uses). To delete a ghost variable, one should just set the number of uses to 0. In all cases, modifications are about a single ghost variable, thus, they are stored in a map, whose keys are variable keys. For each ghost variable, there can be at most one modification. Indeed, updates can be merged according to the following (non-commutative) rules:

-NewKey(key, _, _) + NewKey(key, _, _) ⇒ absurd: one cannot create a variable twice; -NewKey(key, role, _) + SetUse(key, uses) ⇒ NewKey(key, role, uses) -NewKey(key, role, uses) + AddUse(key, delta_uses) ⇒ NewKey(key, role, uses + delta_uses) -SetUse(key, _) + NewKey(key, _, _, _) ⇒ absurd: one cannot create a variable that already exists, or update the number of uses of a variable that does not exist yet; -SetUse(key, _) + SetUse(key, uses) ⇒ SetUse(key, uses) -SetUse(key, uses) + AddUse(key, delta_uses) ⇒ SetUse(key, uses + delta_uses)

-AddUse(key, delta_uses) + NewKey(key, _, _) ⇒ absurd: one cannot create a variable that already exists, or update the number of uses of a variable that does not exist yet; -AddUse(key, _) + SetUse(key, uses) ⇒ SetUse(key, uses) -AddUse(key, delta_uses1) + AddUse(key, delta_uses2) ⇒ AddUse(key, delta_uses1 + delta_uses2) Thanks to the map structure, and since there is at most one modification for each variable, access to the view with updates is really quick, even if adding a modification requires to perform the merge: the cost of the merge is quite small, and accessing properties on variables is far more current than modifying them.

These modifications allow domains to order allocation and deletion of ghost variables. But this mechanism is not almighty, in particular, it is unable to perform ghost reduction. The translator is only interested in the support.

Driver and Combinator

Just as in [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF], to glue domains, we use a binary functor, the combinator, that takes two abstracts domains in arguments and forms a module that implements the same signature as abstract domains. Since it returns a module with the same signature, instances of combinator can be nested, so that, we have a binary tree of abstract domains. The combinator runs the given directive on both domains, starting with the left-hand side domain, and performs the exchange of information for reduction.

But, as explained in [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF], some constraints must be provided to every domain, called broadcast output channel. Thus, the tree is not self-sufficient: we need an additional module at the root to control the iteration in the tree. To broadcast a message, it climbs up to the root, which distributes it everywhere. In the old structure of Astrée, there were no such module, but its job was done by the pointer domain. Here, as we will explain later, iteration is much more complicated and deserve a separate module. This is the driver. It is in charge of performing the reduction as before, but also of enforcing ghost constraints, and acts as an adapter between the struct domain and the composite domain with ghost variables.

As mentioned in section 24.1 "Overview" (page 297), the way ghost constraints are handled is very different from the theoretical description. In the formalization, partial applications of functions U are exchanged between domains, the transitive closure of constraints is computed in each domain and these constraints are run in each domain independently. This strategy has a lot of drawbacks in practice. Firstly, exchanging information in an all-to-all fashion is uselessly costly. Then, each domain computes the same transitive closure of constraints. These two first steps can be centralized: a single module receives all ghost constraint triggering functions, computes the transitive closure and returns it to the domains, which enforce these constraints separately. But, it is still improvable. This strategy does not allow domain cooperation during the evaluation of ghost constraints. Moreover, exchanging partial applications is, to say the least, risky. Though it is often considered as something nice to do with functional languages, it is usually not a good idea for maintainability and debugging. Indeed, when partial applications are buggy, since functions values cannot be introspected, we only know there is a problem when the function is fully applied, which may be a long time after the partial application. Then, when the bug appears, i.e. when the function is fully applied, the context in which the function was partially applied is already out of scope and deleted, preventing us to inspect the state at this point. Moreover, the last partial application is not necessarily the one that prepared the buggy partial application, thus, it is usually difficult and long to come back to the right partial application using the debugger. Overall, in large codebases, long-lived and complex function values are harmful.

In the implementation, we use a very different strategy. When computing an abstract transfer function, typically the application of a unary function (like an assignment or a guard), domains return a new state with a value encoding the progress. There are two kinds of progress values:

-the computation is finished and the progress value holds messages for the output channels (see [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF]); -the computation is suspended, and two additional values are returned:

-a context, a value of a domain-specific type, that allows the domain to be able to resume its computation where it was stopped; -an enriched constraint-DAG to apply. If the computation is finished, the abstract transfer function is called on the following domain, naturally.

If the computation is suspended, the context and constraint-DAG are brought back to the driver, the context is saved on a stack, and constraints in the DAG are run using transfer functions of domains. During this process, computation can also be suspended; in this case, the new context is pushed on the stack and the new constraint-DAG is executed. We proceed recursively, and termination is ensured by the argument explained in section 20.4 "Termination" (page 266). When a constraint-DAG is fully executed, the parent instruction is resumed, using the context on the top of the stack. The domain that suspended its computation use its context to remember its state. It may carry its execution until the end, or generate a new constraint-DAG with a new context. Each domain is responsible to generate finitely many constraint-DAGs.

With this execution strategy, enforcement of ghost constraints is synchronized across all domains, which allows domains to cooperate between each step of ghost reduction, improving the precision. We avoid the exchange of function values and computing several times the same transitive closure. Moreover, this is more powerful than the formalization since it allows domains to emit successive constraint-graphs, by inspecting the state resulting from the application of the previous ghost constraints.

As explained before, for better performance, directed constraints are run as assignments. Constraint-DAGs also include guards, and several other instruction linked to the handling and collection of errors, hints communicated across domains and the abstraction of weak updates. For convenience, a special kind of constraint-DAG is implemented as lists of constraints, to encode chain DAGs. They are sufficient in most cases.

Though execution of ghost constraints is synchronized between domains, all previous hypotheses are still important to ensure soundness. Indeed, domains may not see the constraints in the same order. Let us remember the architecture shown on Figure 24.1 "New structure of the composite pointer domain of Astrée" (page 298). Let a be a constraint being executed in each domain. The topmost combinator run it first on its left domain. The bottom-most combinator does the same. We assume that the numerical domain terminates its computation directly. Then, a is run by the base-offset domain, and let us assume it yields a ghost constraint b, with some context. They are forwarded to the driver that saves the context and run b. It is executed from left to right, and we assume no domain return additional ghost constraint. The evaluation of b is finished. Now, we resume the evaluation of a in the base-offset domain, thanks to the context it yielded. We assume it carries its execution without new constraints, and so does slices domain. At the end, a and b have been both enforced on all domains, but the numerical domain saw a before b, while the slices domain enforced b before a * * . Thus, even with this execution strategy, it is important to allow constraints to be run in an arbitrary order. Moreover, the order of ghost constraint is very depend on the position of each domain in the tree, and the correction should not depend on these positions † † . There is another optimization that is done in the implementation. We remember that an assignment x := e assigns to all variables in x ↓ (the set of variables ghostlier than x), and then may generate directed constraints of the form y ← f where y is ghostlier than x (that is y ∈ x ↓ ). These two steps are reduced to a single one: rather than assigning and then applying a directed constraint (thus, an assignment) on the same variable, we directly run the second assignment, skipping the reset to . This is correct since x does not appear in e, thus the tree of variables under x is disjoint from the forest of variables under variables in e.

This last hypothesis is central in a lot of arguments we made, but of course, it is not true in regular software. Most C codebases contain many statements like i++;, which transgresses this sacred law. The solution is very straightforward, that already as been mentioned in section 19.2 "Semantics" (page 241). We simply need an intermediate variable tmp and perform tmp = i + 1; i = tmp;. If needed, this translation is dynamically performed by the driver using a special ghost variable that is not under any real variable: its role is a nullary constructor. The driver is also able to perform this sanitation on ghost constraints if needed, even if it is not useful for now, it may become useful with more complex ghost structures.

The driver is also in charge of support unification: as explained in section 20.5 "Binary Operations" (page 271), it unifies layer by layer the set of living variables. More generally, all operations that must be performed on all domains are ordered by the driver. to all domains, and they should compare only these variables. There are two reasons. Firstly, this improves performance by restricting the set of variable to examine. But, most importantly, this is a precision issue. Even if it exists in both a and b, if a ghost variable is not to compare, it should not be compared, since it might be unused. And if we add extra (and useless) hypotheses, we will reach an inductive invariant more slowly, and the result will be less precise since it required more widening steps.

This process is coordinated by the driver and relies on the bona fides of domains.

Domains

Domains in this framework should implement a larger signature than usual domains. For instance, support unification functions are specific to the use of ghost variables. Moreover, some standard functions may behave differently. For instance, the function that applies unary directives (like assignment or guards) may returned an unfinished computation.

Because of these characteristics, writing a domain from scratch in the new framework is not a trivial matter. To simplify this task, we provide directions to guide the developer, and utility libraries to ease its work.

Resumable Transfer Functions

When unary transfer functions emit ghost constraints, they return a context to be able to resume their execution. Moreover, we should ensure they do that finitely many times.

Many transfer functions do not need to emit constraints, but when they do, The general strategy is to use an acyclic transition system with finite paths. In practice, we use acyclic finite-state automata (FSAs) with finite memory to be able to keep results between each computation step. The state, with its memory, is a constructor of a sum type, and each computation step perform a transition. If the state is final, then the computation is completed, otherwise, a constraint-DAG is emitted.

We did not manage to produce a usable framework to easily encode transitions, as we did with the microcode for CPU instructions (see subsection 16.1.3 "Microcode" (page 197)). This is due to the large variety of transitions used depending on the domain or the kind of instruction. Developer libraries general enough to take this variability into account did not make things easier to write.

While transitions are difficult to generalize, we still provide an iterator for finite state automata. Indeed, sometimes, it is more natural to build the graph incrementally, for instance, when working on sliced variables, we generate a chain of sub-constraint-DAGs, each about a slice. But when no slice requires ghost constraints, the global constraint-DAG is empty. The provided iterator can detect empty constraint-DAGs (under its different implementations), and will directly run the next transition without suspending the computation, since there is in fact no constraint to emit.

Numerical Domain Adapter

Writing a domain that is able to handle ghost variables is not a walk in the park, either. There are many things to implement and some of them may be quite tricky. It would be a shame to be subjected to such trouble when writing a domain that has nothing to do with ghost variables. For this reason, we have written an adapter that lift an ordinary domain that needs no ghost variables (like domains written for the old framework), to a domain which handle ghost variables (but still does not use any). This adapter allows all existing numerical domains to be reused for free.

The theoretical aspects have been exposed in section 23.1 "Plain Old Numeric Domain" (page 293). But there is an important point to precise. In the comparison process, the driver collects ghost variables to compare, and communicates this set to all domains. But numerical domains have not been written to compare only a subset of variables: they compare all existing variables, and supports must match. To make comparisons works, the adapter deletes all variables that are not to be compared, which solve both problems at once.

Otherwise, the implementation is a quite direct adaptation of the theoretical explanation.

Domain Developer Libraries

To ease the developer work, many internal libraries have been developed to handle various kind of tasks. Some developer libraries are very specific to Astrée implementation. For instance, translator's views and modifications can be used as a single value from a module implementing a signature very close to relevant parts of the signature of the translator. Likewise, abstract states, communication channels and translator's views are all about a single concrete state and are thus always handled together not to risk mixing them. To handle these compound objects, we also provide a rich library to reduce as much as possible the need to look inside. For debugging purpose, we also provide pretty printers for all defined types.

We also defined a lot of more general internal libraries from pieces of code that was recurrent in domains or believed to be useful for the future. They cover a large variety of categories, e.g. memoization, data structures, algebraic computations. . . . External libraries and tools will be presented in Appendix B "By-products" (page 381).

Capabilities

Though we use a combination of very heterogeneous domains, we cannot use arbitrary combinations: some domains are required to make Astrée work properly. For instance, while the interval domain is often very imprecise, it has an opinion on everything and is used as based of all abstractions. Another example is the need of base-offset domain. To resolve dereferences, the struct domain should know variables pointed by an expression. Moreover, there is no special value in this case: set of bases are handled explicitly (since the set of all bases is finite and reasonably small), thus even to return the most imprecise result, the domain should be aware about existing bases. Thus, we need such a domain (base-offset, in our case), otherwise, some operations are not implemented. In fact, requirements should not been expressed in terms of domains, but from the point of view of features: a domain can be able to fulfill several tasks, and conversely, a task can be achieved by several domains. To take these requirements into account, a global set of capabilities is defined, and domains specify capabilities they have.

This way, when domains are instantiated, the driver can check that all required features are fulfilled. Otherwise, the driver immediately raises an error at instantiationtime, thus, before the beginning of the analysis. In addition to required capabilities, we define a set of optional capabilities. They can be useful for domains to enable advanced features that are useless if some featured are not provided by another domain.

Lastly, this is useful for the driver to know if there are domains that use ghost variables without roles. Indeed, if all ghost variables have roles, some optimizations hold. Otherwise, they should be disabled as they would make the algorithm incorrect. For example, if all ghost variables have roles, there is no need to sanitize ghost constraints: as long as the real assignment does not use the left-hand side in its right-hand side, operands of directed constraints are always disjoint. But when a domain allows ghost variables without roles, ghost constraints may need sanitation. In this case, we need to use a stack of temporary ghost variables (for each level of ghost constraints that needs sanitation).

Communication Channels

As explained in [START_REF] Cousot | Combination of abstractions in the ASTRÉE static analyzer[END_REF], input channels are implemented as record types of functions: we need a record type since there are multiple values to hold together, and we need functions fields, rather than explicit maps, since they allow lazy computation. We can go even further and memoize these functions to improve performance. Memoization is usually done with functions as shown in Listing 24.3. But that hides entirely the underlying f function and t table. When functions are enriched gradually by successive compositions, like in the case of input channels, we end up with many intermediate tables that may not be useful anymore. For this reason, we implemented an incremental memoization library that keeps f and t visible. It supports CHAPTER 24. IMPLEMENTATION several memoization strategies to keep caches reasonably small while minimizing cache misses. This library also makes easy to memoize an arbitrary number of parameters, with different strategy for each of them.

Since we work with sound abstract properties, when we enrich input channels, we obtain a more precise sound abstract property by computing the intersection. Defining a function as the intersection of several functions is very common method to get better precision. Our memoizer is well-suited for this application as a memoized function value can contain a list of functions, and the cache is updated incrementally, so that even partial memoized computations, from the time when the list of functions where shorter, can be completed and updated, without redoing everything. To do so, the memoized function should know the default value (that is ) to be returned when the list is empty, and the intersection function to refine results. Moreover, in some cases, we know when computation cannot be improved any more (or does not need to). For instance, for the question "may pointers x and y be aliased?", default (most imprecise) value is "yes" (or tt), the intersection function is the logical conjunction (∧). But if a domain certifies that no, x and y may not be aliased, then it is definitive, and nothing will change the result. Symmetrically, for the question "are we sure pointers x and y are aliases?", the default value is "no" (or ff ), and the intersection is the logical disjunction (∨). Indeed, if a domain can certify that two pointers are indeed aliases, it is definitively true. In these two cases, once we got the most precise value (ff and tt, respectively), it is useless to try to refine the state anymore. Our memoizer also support this optimization that allows short-circuiting a suffix of the list, if the optimal value was reached.

Sadly, some components of input channels may not have a clear value. It was already mentioned in section 24.7 "Capabilities" (page 308) that the set of bases pointed by a pointer expression is given explicitly and thus can only come from a domain aware of such considerations. For such a function, the memoized function object can have a special value that means it is undefined yet: calling it is an error. We can guarantee that a regular value is eventually provided by requiring an adequate capability. It is indeed important to use a special value and not a dummy function (that will always raise an exception, for instance), because the behavior of the enrichment is only a substitution if the field is the dummy value, while it is an intersection otherwise.

Capabilities do not solve the whole problem: to an input channel, a single enrichment request to domains might not be sufficient. Indeed, some functions may have the special dummy value by default, and they can depend on each other (in an acyclic way). Thus, functions that depend on still-dummy function cannot be initialized properly: we need to iterate over all domains until all fields are filled. This is conceptually easy, but requires some implementation tricks to work well with function of various types ‡ ‡ . Because this kind of iterations may be dangerous, along with capabilities, this process is watched to ensure it terminates: the number of unfilled fields should decrease at each step, otherwise there is a bug. In practice, the number of necessary iteration is only 2, but it depends on the order of domains in the reduced product. A not as smart order may need more steps to get the same result: the point is to minimize the worst-case number of edges in ‡ ‡ Currently, it is solved wrapping first class modules in a GADT. 24.9. LOGGER 311 a path of the dependence graph that go from a domain to the right to a domain to the left.

Logger

With ghost variables, bugs are more common, and they often have visible consequences a long time after the actual error. To help debugging domains, we provide a logger. It is a unary functor that take a domain in parameter and return a module of the same signature. It simply forwards all directives to the underlying domain, and returns its results, but with rich printing of parameters and results. This is achievable since we defined printers for all types that appears in signatures (concrete and abstract types), and printers are part of the signature of domains.

Because it preserves the signature, the logger can be put anywhere. By default, there is only one instance, under the driver (see Figure 24.1 "New structure of the composite pointer domain of Astrée" (page 298)). In desperate times, we can make an intensive use of loggers, as shown in Figure 24.2, where we use 5 instances of the logger. This is quite easy since the logger can detect its number of instances and identify themselves, so that the log file is quite readable.

Even with a single instance of the logger, and when analyzing example of a few C lines, it can produce a log file of hundreds of MiB; any small but non-trivial example logs several GiB. To reduce log file to a human-readable size, the logger can be enabled and disabled using annotation in the source code, so that we can log only the section of code that we desire. The other benefit is execution time: I/O are slow; with logging activated everywhere, a very short analysis (less than a second) takes several minutes.

Thousands of hundreds of lines of log can be boring to read, and it is difficult to see at first glance important information, separation between steps. . . . For this reason, we developed an external library: OColor. It makes use of semantic tags of OCaml's Format standard module to produce smart and nice pretty printing using ANSI § § escape sequences. See section B.1 "OColor" (page 381) for more details about OColor. § § American National Standards Institute

Comparison Theory vs. Implementation

As we explained, the implementation is not a direct adaptation of the formal description. Though they match with respect to many aspects, there are some differences. As a rule of thumb, when there are some profound differences, the implementation is more expressive or precise than the formal framework. The implementation of the new framework has been designed to be easily improved, for instance with ideas explained in chapter 27 "Improving Product with Ghost Variables" (page 331).

The first difference we emphasize does not change the expressive power, but can be confusing when working with ghost variables. In the formal framework, an assignment reset underlying ghost variables to , then directed reductions refine this sound, yet imprecise, state. In the implementation, this is compacted into a single operation. Even if it is clearly equivalent, one should keep this difference in mind when implementing.

The evaluation strategy is also synchronized, while there is no such hypothesis in the formal framework. This is not only because mathematics describes the computation 24.10. COMPARISON THEORY VS. IMPLEMENTATION 313 in a more abstract way; synchronization of ghost constraints allows domain cooperation between each step, which is not allowed in the formal framework. One should be careful with these reductions: they should only be about ghostlier variables than those involved in the last constraint. Indeed, overlaying constraints are not yet run by every domain, and the state is consistent only with respect to ghostly enough variables. As we empty the stack of recursive ghost constraints, eventually the abstract state is consistent with respect to all variables.

The current implementation of the reduced product with ghost variables allows domain to emit successive constraint-DAGs. Each constraint-DAG can depend on the state obtained from the previous constraint-DAGs. In the formal framework, domains are only allowed to emit a single constraint graph, that only depends on the constraint and the immediate postcondition of the transfer function (without refinement). This feature is marginally used as we write these lines, but it was judged to be probably interesting for more symbolic domains, for which the transformations induced by a constraint-DAGs may be difficult to predict and thus, new reductions can be useful.

Roles have a slightly different meaning in the implementation and in the mathematical description. Especially, semantics of unary roles is less precise in the implementation, but roles of arbitrary arity are allowed. While, some non-unary roles can be useful (see sanitation in section 24.4 "Driver and Combinator" (page 303)), they have no direct equivalent in the formal world. Each use of such roles should have its own translation in the mathematical framework, needing some tweakings of the semantics to add artificial machinery. Roles with two or more parameters need extra care because they break the forest structure the set of ghost variables has. We need to reinforce the hypotheses ensuring that the left-hand size of assignments does not appear in the right-hand side: since we work with DAGs and not only trees, we need the separation of underlying DAGs, which may be more tricky. Moreover, one should not join both DAGs during such an assignment. A natural property would be to enforce that DAGs under distinct real variables are disjoints; or, even stronger, to require that roles with multiple parameters can only use sibling ghost variables, since it would keep a rigid notion of depth for each ghost variable. rebuild pointers. The same holds about the size of the segment ("limit" field). This is exactly for what the slices domain was designed.

All segmentation policies are not equally reasonable, some may not work at all (see subsubsubsection 2.3.2.2.2 "Segmenting the Linear Address Space" (page 21) and subsubsection 2.3.3.4 "Translation Lookaside Buffers" (page 28)). In fact, we restricted even further the kind of segmentation our analysis accept, because we know what we expect. This is possible since segmentation is a very static strategy: segments are set up at startup and never modified afterwards * . This method would not work with paging which is much more dynamic and proceed to a much more complex address translation. We explore some ideas to prove correctness of paging in section 26.1 "Paging" (page 327).

Slices domain was indeed designed to handle bitwise operations involved in segmentation, and perfectly fulfilled its purpose. It managed to prove that segments are indeed set up according to a variant of the flat model, explained in subsubsubsection 2.3.2.2.2 "Segmenting the Linear Address Space" (page 21) and subsubsection 2.3.3.4 "Translation Lookaside Buffers" (page 28). Shortly said, this method uses trivial segments (starting at 0 and 4 GiB long) for system tasks, and shorter segments for user tasks. Thus, the space at the end of the linear address space is accessible only through system segments. Consequently, we can make these pages persistent in the TLBs to speedup translation, while ensuring no collision with user pages.

A very adjacent problem is the question of GDT entries unlinked to segmentation, like call gates (see subsubsection 2.4.2.3 "Inter-Privilege Calls" (page 31)), and IDT entries (see subsection 2.4.3 "Interrupts and Exceptions" (page 32)). These entries store function pointers in pieces along with various flags (like access rights). Though they are also set at startup, these entries are of a slightly more dynamic nature than segmentation. Their content may be changed after initialization, but, more importantly, we do not exactly know what to expect: fields are not constant values, but slices of functions pointers we do not exactly know. Nevertheless, the process is quite the same. Thanks to the slices domain, abstracting the IDT is basically reduced to the abstraction of an array containing function pointers.

Overall, entries of the GDT are precisely abstracted, and we succeed to retrieve interesting information when needed.

Pointer Alignment

Though the slices domain was made specifically for operations linked to memory protection features, it handles bitwise operations quite generally. Indeed, it was shown to be useful for other tasks that require precise abstraction of bitwise operations. The most notable one is pointer alignment.

Pointer alignment is the propriety of pointer to be a multiple of the word size. In some architecture, like old version of ARM and most RISC (reduced instruction set computer) architecture, pointer alignment is crucial as unaligned access are forbidden.

SUCCESSFUL ANALYSES

317

In other architectures, like x86 or more recent ARM, there have hardware support of unaligned access, though they often require two memory accesses, and thus are slower. However, even in the very lenient x86 architecture, some structures need alignment † . For these reasons, pointer alignment is often an important matter; this is mostly handled by the compiler in usual cases, but for low-lever software, it is usual to bother about it, and having to enforce alignment manually.

In all reasonable architectures ‡ , word sizes are a power of 2. Typically, 32-bit values on 32-bit architecture need to be aligned on 4 B-boundaries. The interesting consequence is that alignment can be checked and enforced using bitwise operations on pointers. An address aligned on 4 B-boundaries is a multiple of 4, thus it ends with two 0. Given a pointer p, we can check if it is aligned by testing whether !(p & 0x3), and we can align it to the previous boundary using (p & ~0x3).

There are two kinds of code we are interested in: those that require inter-domain cooperation, and those that do not. About the latter, let us look at Listing 25.1. At line 2, slices domain manages to remember that the 2 last bits of p are zeroes, which is exactly enough to prove the condition of line 4. In this case, there is no need for domain cooperation. This is interesting, as it was useful in real source code, but very limited as it does not allow arithmetic operations on p, or dereference. To do better, we need the slices domain to emit a constraint to improve the state of another domain. In this case, when we clear the k lower bits of a pointer p, then the new value is in the interval p -2 k + 1, p and p ≡ 0 2 k . Thanks to these pieces of information, numerical and base-offset domains can deduce interesting facts, that allow using the aligned pointer interestingly.

These reductions have been tested in a temporary quick-and-dirty implementation to handle such pointer alignment-related statements in an industrial low-level source code § running on a RISC architecture. These instructions are useful for processor's caches management. † Especially, structures involved in memory protection need to be aligned.

‡ We are never safe from an exotic architecture made by an eccentric electronics engineer. § This was not from the same application as the host platform we use as common thread from the beginning.

Retrying Old Analyses

Since the new framework for domain cooperation is supposed to subsume the old one, and domains have been adapted, it is reasonable to expect that analyses that used to work this the old domain are reproducible with the new domain. In particular, analyses that used to be successful should still produce no alarm, but also, analyses that used to raise alarms should still produce analogous alarms.

After a thorough test-and-debug process on an extensive collection of tailor-made examples, designed to test every feature gradually (since it cannot be made independently), we indeed tried to rerun old analyses. This has two purposes: improve our confidence in the new implementation and evaluate the performance. For both reasons, we used examples of significant size. The performance aspect will be detailed later, in section 25.3 "Performance Evaluation" (page 320).

Rerunning these analyses exhibited a few unknown bugs at the beginning, hard to reproduce on smaller examples. The logger was most useful with such big analyses, especially the activation through annotation feature. Moreover, most bugs were not subtle soundness issues, but important unverified invariant; thanks to sanity checks, they were quite easy to locate.

Once these bugs were fixed, we were glad to notice that the analysis reports indeed matched: we get similar alarms and invariants. Invariants are quite hard to compare: since we use ghost variables, the same property is expressed differently and comparing them requires the expertness of a human eye. Alarms also may not exactly match, but not only because there are ghost variables involved. Indeed, in the old stack architecture, the base-offset used to have a single ghost variable to represent the numerical value and the pointer offset. Now, with the new architecture, the numerical value do not need ghost variables, and the ghost variable is used only to store the offset. The analysis is thus a bit more precise. In practice, it does not allow removing false alarms since this is only useful when performing pointer specific operations (like dereference) on variables that may contains a pointer value or an integer, which is anyway buggy. But alarms with the new version of the base-offset domain are more informative, as we see the offset separately, making easier to track the source of the bug.

Overall, even if the new framework was not used as intensively as the old one (which is in the commercial version of Astrée), we managed to reproduced known results, increasing greatly our confidence in the new framework.

On the Study Case

With this new framework, and thanks to the support of assembly as described in Part III "Abstraction of Mixed C and x86 Assembly" (page 173), we were able to analyze some parts of the study case. We recall that it is an OS hosting several programs. Some are provided by the manufacturer (our industrial partner); they undergo a strict verification process ¶ , and are considered trustworthy. The other applications are installed by the ¶ Probably as strict as the OS itself, maybe using Astrée as well.

ON THE STUDY CASE

319 customer, and the manufacturer has very little control over them; we cannot trust them a priori. We expect the OS to ensure memory isolation so that untrusted tasks cannot interfere with trusted ones. We are also interested in other similar properties, all about isolation, but many of them are not directly performed by the operating system. For instance, we do not want an untrusted task to use the "chown" system call to get the ownership of a file belonging to the system. The implementation of chown is out of our scope, but we still want to prove that the system call mechanism is correct: it is protected against illegal usages, it forwards correctly the parameters to the correct function. . . . We did not directly perform this analysis: for confidentiality reasons, we did not have access to the real code. Our industrial partner gave us representative snippet of codes from which we build the solutions presented before, and performed itself the analysis using a binary of our development version of AstréeS that we provided. These snippets may be simple excerpts, or forged examples that illustrate a specific difficulty. Moreover, we did not aim to analyze the whole OS. As explained before and detailed in chapter 26 "Proving Low-Level Properties" (page 327), there are several aspects that we cannot prove for now (typically correctness of paging).

The OS is separated in a few major parts. The first is the initialization: it begins when the boot loader * * launches the OS, and ends when the OS launches its init program. After the boot, this part is never executed again. Other major parts are about the handling of interrupt and exceptions. We can highlight a few particular handlers:

-the handler of the interrupt triggered by the timer that calls the scheduler; -the handler of the page fault exception that calls the page manager; -the handler of the interrupt used for system calls, which may also calls the page manager for memory allocation and deallocation. We were mainly interested in the initialization phase. We manage to analyze it almost entirely: everything, but the very beginning and the end. Indeed, the beginning depends on the boot loader, on which we know very little. Moreover, just after, paging-related setup is performed, which is not handled for now. At the very end, the OS launches its init process. This was not analyzed either for want of hypotheses about the loader, and file system primitives.

On this part, we proved the absence of RTEs, and the correctness of segmentation according to a modified flat model (see subsection 25.1.1 "Segmentation" (page 315)). As explained in subsection 16.1.2 "System Registers and CPU State" (page 196), we check that segments match expected requirements when segment registers are loaded.

Paging-related code is not analyzable soon as it requires a precise memory domain to abstract page directories and page tables; this is explained in section 26.1 "Paging" (page 327). Indeed, wanted properties (mainly memory isolation) needs a specialized domain, probably from the arts of shape analysis. Nevertheless, we believe that the other aspects of paging-related code can already be abstracted precisely enough. Likewise, the scheduler needs a specific domain, but otherwise, current abstractions should be satisfactory.

We tried to analyze other parts of the OS, mainly the part that handles syscalls. * * A GRUB-like program.

CHAPTER 25. DISCUSSION

We were at first interested in the absence of RTEs, but we failed because of a single infamous trick, worthy of anathema: a memory access to a deep call frame, across a C call frame † † . Some ideas to solve this problem are explained in section 26.2 "Access Across Call Frames" (page 329). Apart from this abomination, it seems that the current state of the analyzer is good enough to perform an interesting analysis, but it is yet to be confirmed, since the remaining functions may hide other surprise, and piece-wise analyses are not always representative of the difficulties of a monolithic analysis, and never as satisfactory.

Overall, we expect to be able to analyze much wider parts of the OS, by developing relatively few domains or extensions.

We were told that the analysis of the initialization of the OS takes about 20 min to complete, using the improved version described in section 25.3 "Performance Evaluation" (page 320). This is considered to be an acceptable duration. We trust that the optimizations described in chapter 27 "Improving Product with Ghost Variables" (page 331) would allow performing the same analysis in less than 10 min.

Performance Evaluation

Apart from precision, the second master criterion is performance. For an analysis to be useful, it should be runnable in moderate time and with reasonable resources. We cannot expect the user to have the goodness to wait for weeks for the analysis of a notso-big program, or to use a supercomputer. The only way to measure performance is to test on realistic software. Indeed, some approaches may have a satisfactory theoretical cost while being way too long in practice, and on the contrary, some algorithms with prohibitive complexity may perform nicely on real-world problems, maybe up to some heuristics. Testing on arbitrary software is not representative. It is indeed easy to write unrealistic source code for which the analysis behaves much worse than in real world cases. For instance, on the new framework we try to evaluate, it is quite easy to undermine the heuristic of ghost variable deletion: for now they are deleted as soon as they become useless, thus it is easy to force domain to spend a lot of time allocating and killing variables. If we kept useless ghost variables, it would be easy to trick them into creating huge, unmanageable supports. In-between solutions can also be trapped ‡ ‡ . Moreover, even without considering adversarial examples, if evaluation cases are not big enough cost linked to front-end (like parsing, typing, translations. . .) are predominant or, at least, non-negligible. Snippets of code written as non-regression and functional tests are in this category and are thus unsuitable for performance evaluation. † † Sizes of C call frames are not supposed to be known.

‡ ‡ The Bélády's anomaly is a good example of how we can make this kind of algorithm arbitrarily bad [START_REF] Fornai | FIFO anomaly is unbounded[END_REF], though we are aware the question is slightly different, this is quite an ill omen for our case.

Results

Unsurprisingly, the new framework is slower than the old one (on code that both succeed to analyze). We will not give raw results as they are impossible to interpret without context, and even with context, it is very difficult and heuristic. Moreover, the general behavior is quite clear and there are very few variations: it is not uncommon to get twice the same timing to the second, for a several minutes analysis. As we do not try to outperform the current state, but understand why it is slower, a general idea is sufficient. In the following, we will denote T the time used for the analysis by old framework, that serve as a reference.

Our first experiment was quite discouraging, but mildly surprising: we get a timing around 70T . Such a slow down is completely prohibitive for an industrial use: a formerly 1-minute analysis takes now more than an hour. A few obvious fixes were applied. First, disabling sanity checks allowed saving most of the time: we went down to 6T by just flipping a few Boolean variables. We went even further in this way, be deactivating at compile time sanity checks and debug printed rather than using a Boolean variable. Even with immutable variables, OCaml compiler seemed not to manage to remove never-satisfied tests. We saved some time and went down to 5T .

The next round of optimizations were less obvious and more specific to this software: it is about roles of ghost variables. If ghost variables have roles, some algorithms are more straightforward, and some precautions are useless. For instance, support unification become much simpler, and sanitation of directed ghost constrains is useless, as long as real assignment are indeed sanitized. Since we only use ghost variables with roles, it would be a shame not to benefit from such optimizations. To do so, there are two solutions: we can dynamically detect variables without role or ask domains at instantiation whether they might create such ghost variables. The latter is faster, as it has a constant (and very small) cost, but the former allows optimizations to be used as much as possible, even if domains may allocate a ghost variable without role in the future. A hybrid approach was chosen, so as to minimize the impact in the code, and not to enforce choices we could regret. To ask if domains may allocate variable without role, we use capabilities (see section 24.7 "Capabilities" (page 308)). These modifications improved running time down to 3T .

From then, there were no simple optimizations with non-negligible outcomes: the current slowdown is still 3. Only precise investigation can allow us to hope for further optimizations.

Investigation

To profile time usage, we used an internal profiling tool, so it can be aware of particularities of the code we are working on, is easier to control and provide more relevant statistics. In particular, we rejected the idea of a global instrumentation since it has been proven to have a major impact on the relative distribution of processing time: adding a constant overhead is prejudicial to very fast functions called many times. Moreover, we are only interested in effective computation time, since it is obviously the bottleneck: there are neither insane memory usage (requiring slow swapping) or other wait on I/O.

As we are performing the same analysis, the exact same transfer functions are called with the same arguments. This simplifies the investigation: we shall find which transfer functions are more expensive than before, by how much, and then find why. It may sound trivial, but this is already a very low-level basis to start investigation: we can ignore anything from the interpreter to the struct domain.

However, while it is simple to find which abstract transfer functions are involved in the slow down, since the internal working may have very few in common with the old implementation, it might be difficult to know what could be done faster.

Given that basis, finding inefficiency sources is not conceptually difficult but is a thankless job, requiring method and patience, with a heuristic part to try to compensate the time spent by using the profiler, which is not negligible for frequently called functions.

Explanation

It appears that there are two major sources of slowdown. The most obvious (and not the biggest) is linked to the increased number of variables. This is part of the big trade-off performance vs. precision. As it is not a net loss (we trade it for precision), and it is not the worse, we judge it to be an acceptable cost.

The other source of inefficiency is binary operations, and more precisely, the preparation. We recall that there are two types of preparation: support unification for operations that return a new abstract state, like union, intersection, widening and narrowing (see section 20.5 "Binary Operations" (page 271) and section 24.4 "Driver and Combinator" (page 303)), and collection of comparable keys for inclusion or equality checks (see section 24.5 "Abstract Order" (page 306)).

Support unification may be inefficient when at each iteration of a loop, we allocate and delete the same ghost variables: allocation may be required by the body of a loop, but the union (or widening) at the top of the loop yield a much less precise abstract state, where the ghost variable is not useful anymore, and thus deleted. At the next iteration, the variable is created again, then deleted, and so on. Doing so lead to an awful waste of time. A solution to this problem is suggested in section 27.8 "Support Heuristics" (page 339). Simply said, the point is to keep useless variables when we believe we will need them later.

The collection of keys to compare is, by itself, quite fast, but implies a subsequent cost. We explained earlier that not all ghost variables are to be compared to check inclusion. This is usually a nice thing, since it allows domains to do less work: they compare only variables explicitly specified. But numerical domains do not work that way. They have not been written for the new framework, but not to lose the quite important collection of domains, we wrote an adapter that promotes a simple domain to a domain able to handle ghost variables (see subsection 24.6.2 "Numerical Domain Adapter" (page 308)). Since simple domains were not made to receive a subset of variables to actually compare, we need to delete all other variables (variables to ignore): the operation that should be faster becomes even slower. Fixes are exposed in section 27.7 "Adapting Numerical Domains" (page 338). The point is to make them to take a set of variables to compare while keeping compatibility, and using another adapter when domains are difficult to adapt. This is not conceptually difficult, the main problem is a software engineering one, to limit the amount of work, and still being compatible with the old framework. Nevertheless, it takes a lot of time to adapt all numerical domains.

Part V

Perspectives and Conclusion

CHAPTER 26. PROVING LOW-LEVEL PROPERTIES

variables, but also the addresses of functions. This requires to get external information usually contained in linker scripts. The most pleasant solution would be to be able to interpret these scripts, but another solution, that already exists in Astrée, is to write annotations on variables and function declarations. Moreover, we need to prove that pages containing all these addresses are identity mapped. Formally, let a ∈ 0, 2 32 -1 be the 32-bit address of a variable. We need to prove that * * (CR3 + 4 × a [22 : 31]) [12 : This is what we need to do when activating paging, but it is not enough to prove that paging works correctly: we also have to check that pages do not designate the same physical memory (except explicitly specified, for shared memory). This is more difficult since each process has its own page directory, but only the currently executing process has loaded its page directory address in CR3: we need to keep track of page directory of suspended processes.

Once again, from a theoretical point of view, suspended processes are not a distinct feature: given the state of a machine, there is no general way to identify suspended processes without knowing conventions and strategies used by the system. The only interesting criterion is that the memory of a process should not be changed while it is asleep, keeping track of page directories is an overapproximation that allows replacing a complicated security property (expressible only on set of traces), by a safety property.

To find page directories, it might be enough to remember which values have been in CR3 register. Though, it does not take into account that processes can be terminated, and thus its pages can be reused. A solution might be to consider pages as unused, until proven otherwise. A page that belongs to another process, currently suspended or terminated, can be given to the current process. The error will be emitted the next time the suspended process is scheduled again, that is, when it loads its page directory base address into CR3: this is the proof that it was not terminated and that the page should not have been given to another process. This method does not need annotation or heuristic to guess where are suspended processes and when they are terminated or just asleep.

At last, there is the problem of abstracting paging structures: page directories and page tables. They form almost a forest from page directory bases addresses to pages, but some converging edges may appear when sharing memory. Abstracting such structure will probably require shape analysis methods.

Overall, abstracting paging is a huge work. There are plenty of difficulties, and we probably just skimmed the surface of the underwater peak of problems. Likewise, solutions that we have mentioned are probably a very small part of the work. Also, a possible future implementation in Astrée would be one of the biggest changes. Nevertheless, it is almost sure that the new framework for domain cooperation with ghost variables and 26.2. ACCESS ACROSS CALL FRAMES 329 partial support of assembly will be helpful.

Access Across Call Frames

In our target application, in some parts of the code, there are unsupported operations. Though there are few of them, they prevent to analyze these parts. One of these unsupported operations are access to a deeper call frame in the stack. Handling such accesses would allow us to analyze crucial parts of the code.

In our handling of mixed C and assembly, we only allow assembly blocks to access global and current local variables. However, sometimes, assembly is used to dig deeper into the stack, and to read and write local variables of a caller (not necessarily the closest). Our abstraction of call frames indeed reject such accesses (see section 15.2 "Abstracting Successions of Call Frames" (page 188)) but it would be useful to relax this restriction.

The solution seems to be conceptually easy, but difficult to implement. Moreover, we need much more hypotheses beyond the C and assembly standards or the calling convention in use: we need to know how the compiler manages call frames. The goal is to be able to predict the size of each call frame, so that we know how the offset needed to skip them. This is sufficient to handle access to an assembly stack segment, but if we try to access a C local variable, we also need to know where local variables are inside each call frame. These are very strong hypotheses on the compiler but there are not superfluous: without them, a code that accesses deep call frames is illegal. When we use such hypotheses, the analysis becomes specific to the compiler, and a priori, it would not work with another compiler. This is not a weakness of the analysis: it is the same in the concrete world: compiled with a compiler that manages call frame in another way, such a code will not work. Short, we cannot expect an analysis to be successful with fewer hypotheses than we made when writing the code, and thus, hypotheses we need to compile it.

Though the idea is simple, there are a few difficulties to implement it properly. First, from a very technical point of view, as Astrée was designed to analyze C, it assumes no relative position of variables, even local ones. Moreover, it is not clear how to encode these hypotheses. They are very specific to a given compiler, and thus, they should not be hard coded in Astrée. Rather, it would be better to write them in configuration files interpreted by Astrée. Such files should contain hypotheses related to call frame management in a general enough format. Then, we would be able to write the call frame management policy of any (reasonable) compiler, or at least a relevant fragment, to test if the code is still correct when compiled with these hypotheses.

Chapter 27

Improving Product with Ghost Variables N ew domain cooperation framework is already a big change that provides a great expressive power, but it still can be improved in many ways. Obviously, we may add new domains, but also we can improve the framework with new features (useful for new domains), and try to improve the concerning performance issue presented in section 25.3 "Performance Evaluation" (page 320).

We give some improvement possibilities, which are not light to implement or formalize. We start by idea about new features or domains, and we give after the suggestions to improve performance. Most of them are very general ideas that require much more work.

Abstracting a Different Concrete World

Current version of the product with ghost variables is designed to be very general with respect to the semantics of the language, but requires memory states to be maps from variables to values, without aliasing or dereference. This was useful to justify that righthand side expression are not changed when assigning the left-hand side with directed constraints. If modifying the abstract state of the left-hand side impacts the value of the right-hand side, strange interference may happen.

Nevertheless, most languages, especially C, allow pointers and dereference, even if sometimes it is only implicit (most high-level languages use pointers implicitly). It might be interesting to handle ghost variables in a world where aliasing exists. In the stack of domains of Astrée, that would mean to lift it before memory accesses are resolved into physical cells (above struct domain or at its place).

Since we cannot have a syntactical guarantee that assignments will not interfere, we might crave for a more subtle criterion: a semantic argument. It should be possible to adapt the ghost variable framework for any state that may be split into independent parts. For instance, it could be a way to implement cofibered domains [START_REF] Venet | Abstract cofibered domains: Application to the alias analysis of untyped programs[END_REF]. Typically, we could imagine using several domains of shape analysis, side to side: node of each 332 CHAPTER 27. IMPROVING PRODUCT WITH GHOST VARIABLES domain can be a substructure of another one. With a domain that handles trees, and one that handles lists (with a domain to store numerical values), we could represent lists of trees as well as trees of lists, and any recursive combination. We rely on domains to prove that there is no harmful aliasing dynamically. We could still default to if something goes wrong.

A similar architecture could be use at the supreme level: iterators. We could imagine an adaptation of the framework where iterators are at the same place as ordinary domains, and not above, as usual. The product is fed the whole program (for instance, as an AST); while most domains have nothing to do with directives more complicated than a comparison or an assignment, iterator-domains can interpret the program and translate each syntactical element into abstract directives sent to all domains. This may be interesting to implement gradually a language: each syntactic or semantic feature can be handled by a different domain. If there is no domain to provide constraints for a given feature, the default value remains, which is correct. We may also raise an informative alarm to explain that a non-leaf node of the AST were not handled. Making each language feature handled by a different domain may allow simplifying the maintenance, but also to reuse elements common to several languages. From a theoretical point of view, we believe it requires not that many adaptations to formalize. Indeed, we could see iterator-domains as translator of constraints: constraints are initially written as a part of an AST (which is a constraint thanks to the language semantics), and the iterator generates an equation on abstract states. This is close to the intention of [START_REF] Journault | Combinations of Reusable Abstract Domains for a Multilingual Static Analyzer[END_REF], which simplify gradually the programs and expressions. The solution we propose do not enforce an ordering in the simplification; our hierarchy is completely flat.

Moreover, we are not limited to have a single domain for each syntactic feature. For instance, we may have several translators for loops. For range-based for-loops (for in loops), several translations are possible. In C++, such loops may be desugared using iterators, but if we have a higher level description of the data structure on which the loop iterates, we may generate inductive predicates that would work very well with a specialized domain, like a shape analysis domain. Even standard while-loops can be translated into internal directives naively using widening at loop's head, or using a finer widening strategy, or even unrolling. Correctness criterion is simple: each translation must preserve (or overapproximate) the semantics. But there are other problems, we name two.

-How to ensure termination when some translations can be applied arbitrarily many times? For instance, loop unrolling generates new unrolled statements, but a modified loop (for non-unrolled iterations) is also emitted. It is still possible to unroll this new loop. -How to choose the next translation when there are multiple possibilities? For the first problem, we can imagine there are two (or arbitrarily many) instances of loop-unrolling domain. Indeed, a single domain can be smart enough not to unroll its own result. A way to ensure termination is to annotate each statement * with its history.

GHOST VARIABLE RENAMING

333

In this case, the history is a tuple where each component is managed by a domain: each time the domain transforms a statement, new statements inherits the parent history, but where the component corresponding to the current domain is modified. Each component should follow a well-founded order defined by each domain, which ensures termination. In the case of loop unrolling, the component can be an integer representing the "fuel" of the domain: how many times it still allowed to unroll the loop. Another interesting example is the (rather trivial) domain that transforms a sequence of statements into the composition of the semantics of each element of the sequence. Since syntactic construct (such as branching or loops) may be arbitrarily nested, the sequence-domain can be called arbitrarily many times to obtain elementary statements. A solution is to set its fuel to the depth of the AST, and each time it splits a sequence, the fuel is decremented in newly generated statements. The fuel can never be negative, ensuring termination. Moreover, if the AST contains non-sequence nodes on each path to the deepest leaves, the fuel will not even reach 0.

The question of choosing the transformation is more heuristic. Each translation is sound and a single translation can be precise enough. To favor one over another, we may rely on heuristics to guess which one will be the better one. But we may still be wrong, and performing the analysis again, but with another translation will give another sound result. The intersection of both results will be more precise and still sound, but it might be unnecessarily long. An idea could be to annotate the analyzed program with important points for each domain: domains can say where they can be useful (the loop node, for loop-related domains) and where we will get the result (after the loop). If we choose a translation and the remaining of the analysis succeed, we do not need additional work. But if it fails, a backward analysis (such as a causal analysis) may guess that the loop's postcondition is too imprecise, and try to improve it using another transformation and reanalyzing the loop. Hopefully, it will not be necessary often, since we believe that in many cases, the right translation is not difficult to guess (depending on the traversed structure, the language, the kind of the loop. . .).

Of course, these are only general ideas, and an actual implementation would probably raise many other problems. However, we trust this is a possible solution to build a supremely modular analyzer supporting many languages. The question of performance is probably the biggest problem: such a solution requires much more machinery than the architecture of Astrée, and it would probably have a large impact on analysis time. On the other hand, a flat architecture offers other optimization opportunities, e.g. it makes distributed computations much simpler. Slow domains may instantiated many times and the workload distributed.

Ghost Variable Renaming

Utility of handling linear combination of ghost variables has been explained in subsection 18.1.3 "Linear Combinations of Pointers" (page 234). Yet, as shown in section 23.3 "Linear Combinations" (page 295), ghost variables representing terms of linear combination are not meaningfully ordered. A smart ordering can help to preserve as much precision as possible. Though, it is only possible to find the good ordering once we have both states, to compare how to lose the least precision. Then, we need to rename variables so that the following binary operation will be as precise as possible.

Exchanging roles of variables requires variables to have a symmetric semantics, so that, when roles are swapped, the concretization do not change. To do so, we would need an internal operation to change variable roles. This can be made using a temporary variables and assignments, but it would be faster as a simple relabeling. Such feature would allow implementing a smart linear combination domain.

We could generalize this feature to any rewriting rule that describes a sound transformation. Such a case will be exposed in a following section.

Partial Partitioning

Partitioning is a general and systematic strategy to improve precision, though it is usually not clever and may be expensive. Indeed, most of the time, partitioning is only useful for a few variables, but we duplicate the abstract state in memory. One may argue that persistent data structures allow common substructures to be shared, but it is often a negligible improvement for long-lived objects. Indeed, maps (since they are the common way to store the variable-to-values correspondence) are balanced and need sometime to be rearranged. When it happens, a lot of shared substructures are broken into smaller shareable substructures, at the end, the benefit may not be clear.

Ghost variables may provide a nice solution by implementation partitioning as a part of the abstract state, and not above. Given a partitioning condition c and a variable x, we create variables T rue c (x) and F alse c (x) to store values of x in cases where c is true and false respectively. This allows the analyzer to partition only variables we are interested in.

For instance, initially, we may only partition variables that appear in condition c. After each assignment, the left-hand side variable should be partitioned if there is at least a partitioned variable in the right-hand side. This is the most comprehensive partitioning since other variables are not impacted by the value of c.

Smarter heuristics can be designed depending on the application. We can also implement many subtle improvements. For instance, delete ghost variables if they are equal, or if one of them is strictly greater than the other, deleting the greatest one will not lose precision, since x should be equal to this variable. It may also be useful to end partitioning through annotations: we can manually reduce the lifetime of ghost variables, improving performances. Enhanced rewriting rules can be useful when we partition according to several conditions. Let c and d two conditions and x a variable. Let us assume that we partition x with respect to c and d. There are several possibilities: partition could be flat, as depicted on Figure 27.1, or nested, as on Figure 27.2. Flat partitioning avoid the following problem we are about to explain, but it has less precision, thus it cannot be used in every case, and sometimes, nested partitioning must be used. We assume the current state is as shown on Figure 27.2, but, e.g. thanks to an annotation, we want to end the partitioning with respect to d; this is very easy: we just need to delete variables on the leaves. If we want to end partitioning with respect to c instead, we need to transform the tree to obtain the tree Figure 27.3. This can be done using rules This can also be expressed as a single rule as tree rewriting. From the new tree, we can easily end the partitioning with respect to c. We can remark that ending a partitioning can also be expressed as a sound tree rewriting rule. Both rules can be composed to get a single, more efficient rule. This partitioning domain is limited as it can only work at the same level as the new framework, but it could be an optimization by limiting the size of the abstract states in memory and the number of abstract transfer functions to call. We shall also take care not to partition ghost variables under a partitioned variable; this requires domains to be able to look in the stack of roles.

Sparse Support

Thanks to the previous domain, we might restrict the number of partitioned variables, improving the performance. But each partitioned variable need the original one x and two ghost variables T rue c (x) and F alse c (x), while simply duplicating the state needs only 2 variables. When we partition many variables, this may be slower than the basic implementation.

A solution might be to remove unnecessary variables. Values of some variables may entirely be determined by its ghost variables only. This is typically the case of partitioning: the value of x can be found from the value of T rue c (x) and F alse c (x). This case is very easy as it requires a simple abstract join.

Even if x is required in the enriched concrete semantics, it can be missing in the abstract state. We need to adapt the concretization to generate all variables above a ghost variable, even if it is missing. Missing variables can be rebuilt on demand. One should find a good heuristic so that the deletion and reconstruction of variables do not occur too frequently to ruin the potential performance gain. This can only be designed from a real use case.

Activation on Demand

Since the new framework is slower, if the analysis only uses domains that existed in the old framework, it would be foolish not to use it. In many cases, slices domain is absolutely necessary, but on short sections of code. Based on this observation, we could use the old framework by default, until we reach a point where additional domains with ghost variables are needed. There, the state is exported to a state of the new framework and the analysis can carry on. Once the new framework completed its deed, we perform the reciprocal translation; we can lose information, but nothing still useful.

Adopting such a strategy doubles the maintenance cost as both the new and the old framework are still in use. Moreover, abstract state translation is probably very expensive: it may only be suitable for code where there are few translations to do, and where sections that may be analyzed with the old framework are big enough.

This idea is mentioned to get a quite systematic list of solutions, based on the criterion we may be able or willing to relax, but it has very little chance to indeed work. Moreover, due to the maintenance cost, it may not be desirable anyway. A nicer approach would be to activate domains on demand, while using only the new framework. Sadly, automatically detecting if a domain is required is not that different to actually use it. We could also rely on annotations, but that would mean to lose automation.

Parallel Computations

When we cannot improve the algorithm or the implementation, a classical strategy is to try to make it parallel. Let us explore how it works in our case. This implies two questions: how to divide the work into parallel tasks and how to make these tasks thread-safe.

The first, simplest, strategy is to parallelize computations from a higher level, for instance, from the partitioning domain (see Figure 14.1 "New structure of the composite domain of Astrée" (page 179)). This is easier as we do not need to modify the way the new framework operates, but it still needs a few adaptations. Most of the product with ghost variables is thread-safe as we use mainly immutable data types and pure functions. But, there are few mutable values that should be taken into account. First, we can mention maps used in memoized functions. These functions are thread-local and thus, their hidden maps are thread-local as well. The other mutable data is the state of the key dispenser. It is explained in section 24.2 "Roles" (page 299) why it needs a global mutable state. For these reasons, we understand that the state should in fact be globally shared across all threads. This implies the use of locks. This basic exclusion method would probably not impact the performance in a great extent since modification to the state of the key dispenser are quite rare in average, and very fast. This kind of high-level parallelism is already implemented in Astrée (before the work presented here) but cannot be used with the new framework for the reason explained. If parallelization is eventually supported, we would need to adapt the logger as well: it should not write two lines at the same time, and each instance must identify its thread. Moreover, it would be nice to keep lines about a single abstract directive consecutive. Each logger can use a separate file or use a cache to delay lines until the computation of the current statement is finished and print everything at once. This is more readable but produce no output when a computation does not terminate.

We may also wonder if we can introduce parallel computations in the new framework itself. As a rule of thumb, since we used synchronization to perform more reduction, we cannot expect that much parallelism without having a prohibitive cost in inter-thread communication. A possible improvement would be to make each domain run in a separate thread. It would require to adapt the iteration strategy, in particular, all domains are run at once, and they may all yield constraint-DAGs. A simple implementation wait the slowest domain to complete before running each DAG (and so on, recursively). A more complex implementation may deliver constraint-DAGs asynchronously. In both cases, the possible performance gain is not clear, but it may be worth to try when working with a high number of domains.

Another, possibility is to parallelize only the computation of parallel edges in constraint-DAGs. This would be much easier to implement, but it would require extremely long parallel branches (or a lot of recursively collected constraint-DAGs) to compensate for thread creation and communication time. Once again, this is a very heuristic estimate, and implementation and testing is the only way to have an idea of the real efficiency. Though, it might be heavy to implement for a discouraging result, we do not recommend it as being the first option. Especially, following sections give ideas that would almost surely be not disappointing.

Adapting Numerical Domains

It was said in subsection 25.3.3 "Explanation" (page 322) that a major source of inefficiency is the preparation of comparisons in numerical domains. Indeed, the adapter does the best it can, but since numerical domains are unable to receive a set of variables to compare, the adapter must remove all other variables.

The solution is simple: we have to adapt all numerical domains to be able to receive a set of variables to compare, and ignore the other ones. In most domain, the transformation is easy. For instance, the composite non-relational domain stores the state of each variable in a map from variable keys to abstract values; for these domains, we shall simply iterate over the set of the variables to compare rather than the whole map. Many relational domains would require very few transformations as well. Yet, some domains, especially those that are implemented in C or packed domains, are more difficult to adapt. A temporary solution may be to apply the adapter only on domains that are difficult to adapt. There would be several instances of the adapter, tied together by the combinator of domain with ghost variables.

The performance gain is almost sure, but there are several drawbacks. First, adapted domains would not be able to work in the old framework, which is for now the faster and more reliable one. Moreover, apart from the comparison functions, there are many other functions to adapt to get the right signature, even without using ghost variables. Finally, using the combinator of domains with ghost variables is less efficient than the combinator of numerical domains that does not take care of ghost variables; since we know this subtree of domains does not use ghost variables, the old combinator is sufficient.

We believe there is a common solution to all these problems. Firstly, we adapt numerical domain by adding a new function that allows partial comparison, given a set of variables. This way, the traditional signature of numerical domains is preserved. The old combinator of numerical domains needs the same adaptation, so that we get a composite domain compatible with the old framework, with an additional function for partial comparison. We can write a small adapter for domains for which it is difficult to implement partial comparison. It will work in the same way as the current adapter, by deleting variable we do not want to compare. This small adapter only generate the partial comparison function and can be used on each domain that needs it, and not at the root of the tree of domains. Finally, we change the current adapter (the one at the root) to use this new partial comparison function instead, so that it does not need to remove extra variables anymore.

This solution is probably the best since ghost variable management is entirely performed by the adapter. Underlying combinators are good old numerical domain combinators, which do not work with ghost variables, and thus are more efficient. Domains are still compatible with the old framework, and the change is minimal. This solution has not been implemented because of time constraint and the number of domains difficult to adapt; while there are relatively few of them, the absolute number is quite important, since Astrée has many domains.

There is another way in which old domains may be adapted that allows to improve the precision of other domains. During support unification, one may need to add an unused ghost variable in an operand if it exists in the other (for instance, see section 21.3 "Adaptation in the New Framework" (page 277)). For a join, since this newly created variable is unused, it would be extremely precise to assign it the value ⊥: during the join, only the value of the existing variable is kept. Though, as mentioned in section 21.3 "Adaptation in the New Framework" (page 277), numerical domains implement ⊥-coalescence: when a variable is ⊥, the whole property is ⊥. This used to be relevant when there were only real variables, but this is not true with ghost variables: if an unused ghost variable gets the value ⊥, the concretization of the state is not necessarily ⊥. For now, we store in the new ghost variable the value of the existing one in the other property; while being optimal for non-relational properties, it destroys relations that involve this ghost variable. Adding a mechanism to prevent unwanted ⊥-coalescence, would allow assigning ⊥ to unused ghost variables and, hopefully, keep relations. A solution may be to have a Boolean value for each ghost variable that tells whether it is useful with respect to its immediate parent. When ⊥ appears in a used variable, it propagates to the parent, and recursively until we reach a real variable or an unused variable. If we reach a real variable, ⊥-coalescence indeed applies. This "is used" status is not the same as the reference counting used for ghost variable collection (see section 24.3 "Translator" (page 301)). Indeed, a freshly added variable is not used in a relation (preventing ⊥-coalescence), but is still useful for the domain (not to be deleted). Using the same counter for both notions ("ununsed" and "to be deleted") would imply to delay variable deletion to specific steps, in case a variable may be in fact useful; this prevents deleting actually unused variables between support unification and the following binary operation, whereas variable deletion is an essential part of support unification. Overall, it is better to allow domains to allocate and keep unused variables. Keeping unused variables is also the key idea of the optimization explained in the next section.

Support Heuristics

Another major source of inefficiency is support unification in preparation for binary operations. Many variables are added and removed, though most of them are not new. Indeed, we can check that the key dispenser already provided a key for these roles. In practice, they are mainly variables that exist after the analysis of the body of a loop, but not in the cumulative state at the top of this loop.

Indeed, variables are added by precise computations, then removed when merging with a less precise state. This process is very expensive. A solution is not to remove ghost variables, even if they are unused. This way, we would only add new variables, which will likely converge to a stable support. Yet, this method is not advisable since it would result in huge supports with important memory usage. We must be able to remove variables, but not too soon.

Once again, there is a world of strategies, and once again, there are two main possibilities: user-written annotations or automatic heuristics. Annotations transfer the responsibility of the problem to the user, which makes the analyzer even more difficult to use. Of course, as good as heuristics can be, we can still write code where they fail. That's why we always advocate to use both methods: smart heuristics allow the everyday user to make most analyses successful, and annotations allow the expert user to guide the analyzer when heuristics fail. An even more powerful solution is to be able to describe heuristics in a configuration file using a domain-specific language (DSL). This allows the expert user to automate its work, but within the limitations of the DSL.

We will explain a heuristic that seems both reasonable and efficient. Repeated union, widening and other binary operations happen mainly in loops, as explained before. Of course, support unification is also performed at the end of if-then-else (see section 20.5 "Binary Operations" (page 271)), but only once. The proposed heuristic is to simply not remove unused variables while analysis the loop, and remove unused ghost variables only after the loop analysis. Of course, as any heuristic, its efficiency can only be proven by actual experiments. Real world applications can also help to design new and more adequate heuristics.

Chapter 28

Conclusion

In this thesis we examined the problems raised by the analysis of mixed C and assembly code, motivated by the formal verification of an industrial study case, a host platform, with Astrée. We are interested in the absence of run-time errors, and some security properties (like memory isolation) that are overapproximated by low-level safety properties. We saw there are mainly two parts.

The first problem is the interaction of C and assembly, though they use very different memory model and control flow structures. But before being able to analyze such code, we need to define its semantics. We provided simplified versions of C and assembly, to build a model of mixed C and assembly code. The main point is the memory model that allows C and assembly operations to work correctly together. This semantics requires some hypotheses that are not part of C standard, but which are necessary to reason on mixed code, otherwise, we cannot conclude anything interesting. Parts of our study case need even more hypotheses and a more precise memory model, but we have ignored these parts for now, as the assumptions involved are very specific to this case (and compiler-depend), and are only useful for a few artful, but essential, snippets.

Once the semantics is set up, we can use any method to analyze programs. Since we focused on abstract interpretation, we showed some abstractions that are sound and are precise enough for our study case. The domains that we designed depend on the software we analyze, and even when a program is correct with the semantics we have defined, domains may be not precise enough, and wrongfully find errors. Though, we explained why we believe that the abstractions that we presented are both efficient and precise enough in many cases. Another benefit is that our abstractions are parametric: they abstract an aspect of assembly (e.g. registers) using more generic abstractions (e.g. integer domains). This allows the developer to fully reuse existing domains, and improve simultaneously general and assembly-specific abstractions.

The second problem is not directly related to the presence of assembly, but comes from what we do when assembly is involved. Indeed, if assembly parts could be done in C, we would not bother writing mixed code. Assembly blocks exist for specific purposes (like using CPU low-level features), that imply operations that are pointless in pure C. We detailed two examples: chopping bit slices of values (including pointers) and rebuilding them (typically to fill a GDT, or manage paging), and computing linear combinations of values (e.g. for dynamic code). We saw that these problems could be solved easily using ghost variables, but we also explained why it is not easy to completely use the power of ghost variables in a reduced product of abstract domains. In this thesis we proposed a framework that solves this problem. It allows domains to cooperate (that is, to perform partial reduction), and share ghost variables in a very decentralized manner. With this framework, new domains using ghost variables can be added without modifying anything in existing domains. Conversely, new domains can also improve the precision of existing domains, since the precision of each domain depends on the precision of communication channels. We can develop a rich collection of abstract domains and use only the required subset for each problem to get optimal performance. Thanks to adapters, we are able to reuse existing domains that does not handle ghost variables, which allows the new framework to be at least as precise as the former one.

All these solutions have been implemented in AstréeS, a development version of Astrée, showing that they achieve required precision on our industrial study case, and we trust they are precise enough to be used on many reasonable source codes. Moreover, when a better precision is needed, it might be sufficient to improve the underlying abstractions, without changing the parametric abstractions that are specific to assembly handling. In terms of performance, these solutions are satisfactory and can be used on real-life software, though there is a clear, but not prohibitive, slowdown associated with the product of domains with ghost variables.

Of course, there are some possible improvements. On the first problem, we mentioned that sometimes, additional hypotheses are ensured and required. If we want to take them into account, we need to make the semantics more precise accordingly and find more precise abstractions. Abstract domains can also be improved, like any domain, but it is likely to be a problem not limited to the assembly. For instance, a proof of paging would need an adequate memory model to represent the structures involved (page directories and page tables).

Many improvements are still possible on the product of domains with ghost variables. First, we can always add new domains, according to the needs. The modularity is very satisfactory, and for now, it provides much more features that are used by domains (like ghost variables without roles). Nevertheless, some domains can benefit from even more advanced features (like role rewriting for partitioning or linear combinations). We also explained briefly how this product of domains could be lifted to build a whole analyzer with a supremely flat, thus very modular, architecture. Currently, the main problem of the product of domains with ghost variables is the execution time: while the slowdown may be considered an acceptable trade-off for improved precision and expressive power, it is still significant when the old framework is good enough. We gave several ideas to speed it up; some are internal (like better adaptation of old domains), but other options imply transferring some tasks to the new framework when it can perform them more efficiently, improving global execution time (like ghost variables-based partitioning). We think these optimizations may reduce the slowdown to an acceptable value.

Our ultimate motivation is the analysis of our study case, which is an operating system. The analysis of OSes is very complicated, but necessary since they are a fundamental layer in many software systems. We expect that OSes match very specific properties, like memory protection (which was studied in this thesis) or fairness of the scheduler. Some questions, like the latter, are usually studied separately as they are rich enough per se. Specific analysis tools may eventually be integrated in a general analyzer like AstréeS, to make it able to check entire OSes at once.

D

Notations and Basic Definitions

T his chapter sums up notations and goes beyond by explaining some choices. It cov- ers most topics, from architecture to fundamental mathematics, and recalls classical notations as well as it introduces more original ones. arr [7] arr [6] arr [5] arr [START_REF]1 Dereferencing an arbitrary integer[END_REF] 4 When talking with computers about integers, it is often nice to use a non-decimal radix. Especially, binary and hexadecimal are good natural choices. Most programming languages use respectively the prefixes 0b and 0x. But, to follow Intel's manual's style, we also use suffixes B and H, in particular in a mathematical (non-code) context. For instance, 10000B = 10H = 16.

A.1 Architecture

We also follow Intel's manual's notation of memory structures: bits are placed from right to left (so that number are written in natural order) and bytes of lower address to the bottom. Bit offset is written on the top and byte offset on the right. An array arr of 1-byte values will be arranged as on Figure A.1, using C notation for array accesses.

We also need to cut bit sequences (like integers or addresses) and to rearrange them. To extract a slice of bits, the syntax is s Units and prefixes are a common source of confusion when counting bits. We follow the specification of the International System of Quantities (ISQ) (ISO/IEC 80000) that defines, among other things, the prefix and quantity used in the SI * . The SI defines a proper subset of prefixes with respect to ISQ, but they agree the ones they have in common. However, the SI does not define units and prefixes specific to computer science.

The ISQ defines 3 units we are interested in: bit (bit), byte (B) and octet (o). A bit is a single binary digit, a very fundamental piece of information. An octet is by definition a sequence of 8 bits. A byte is the smallest addressable unit of memory. Once upon a time, bytes were not always 8-bit long. But that time is over. It is now the age of 8-bit bytes. And (not so) recent standards go in this direction. The x86 architecture plays it safe: a byte is indeed an octet. So we use both without worrying too much.

The other problematic point is the correct usage of prefixes. The SI prefixes are not questioned: k (kilo) is 10 3 , M (mega) is 10 6 , and so on. Thus, a kilobyte (kB) is 1000 B. For binary-friendly prefixes we used dedicated ones, as defined in the ISQ. These prefixes are listed in Table A

A.2 Logic

We write a := b to define a as equal to b. We prefer this notation over , or Relational notations can be used in their reversed variant, e.g. A x ⇔ x ∈ A. We can also use crossed out versions to state the opposite e.g. A B ⇔ ¬A B (A is not a proper subset of B).

A less classical set operation we use sometimes is the disjoint union. This is a touchy subject with no perfect solution. The idea is to compute the union of two (or more) sets, without collapsing elements that appears in both. Moreover, elements are tagged so that we know from which of the source sets they come. Like in many cases, they are two philosophies to define it: explicit ("this is how the disjoint union is defined") or axiomatic ("this is how disjoint union should behave"). Category theory has an axiomatic definition of a generalization of disjoint union (the sum or coproduct). It is not very handy and does not define it uniquely. Explicit definitions of A B usually build the set of elements that are either in A or B, accompanied by a tag to discriminate the origin. For instance, we can let

A B := {(∅, a) | a ∈ A} ∪ {({∅}, b) | b ∈ B}
Here, we choose ∅ and {∅} for the tags because they are very fundamental objects, but the choice does not matter at all. This definition is very simple but it has several flaws, most important ones are that this operator is not associative, not commutative and that A ⊆ A B. The latter point force us to explicit the tag; we do not want that.

Overall, there is no good solution that provides a formal definition and that is easy to use when writing math (in particular, that allows to forget about the tag). So, we are deliberately misusing the notations: we take the liberty to write x ∈ A B and then distinguish cases whether x ∈ A or x ∈ B, provided that exactly one of these possibility is true.

Let us also remark that we are using for disjoint union, and not the usual , because we keep the latter for the "join" operator of a lattice.

Let us also clarify the case of intervals. Closed intervals of real numbers are denoted For instance 0, 1 = {0, 1} but 1, 0 = ∅. This is very useful for base cases of recursive formulae, for iterative definitions or limit cases. Let E be a set and R be an order relation on E. Let A ⊆ E, we write A ↑R := {x ∈ E | ∃y ∈ E : yRx}, the set of elements of E that are bigger than an element in A. Symmetrically, A ↓R := {x ∈ E | ∃y ∈ E : xRy}. For convenience, given a ∈ E, we write a ↑R := {a} ↑R and a ↓R := {a} ↓R . This notation can be generalized to any relation, though it would usually be ill-advised. Yet, an interesting and nice case is the case when R is a strict order. equal to f on this subset. Dually, given a set Y , we define the corestriction f |Y to be the function from A to Y ∩ B equal to f where applicable. We have, supp f |Y = {a ∈ supp (f ) | f (a) ∈ Y }. Thus, in general, a corestriction is a partial function. If we make explicit how functions are built, it is even easier (but not as intuitive):

f [a → b] : A B x → b if x = a f (x) if x = a ∧ x ∈ supp (f )
It is simply the function f , but the image of a is overridden to be b. Consequently, supp (f ) ∪ {a} = supp (f [a → b]) (no matter if a ∈ supp (f )). We can iterate this process:

f [a 1 → b 1 ] [a 2 → b 2 ] = (f [a 1 → b 1 ]) [a 2 → b 2 ]
for any (a 1 , a 2 ) ∈ A 2 (with a 1 = a 2 ) and (b 1 , b 2 ) ∈ B 2 . And more generally, given n ∈ N, (a i ) i∈ 1,n ∈ A n such that ∀(i, j) ∈ 1, n 2 , a i = a j ⇒ i = j and (b i ) i∈ 1,n ∈ B n , we define With this second version, we can directly generalize it to any indexing set.

f [a i → b i ] n i=1 = f if n = 0 f [a n → b n ] [a i → b i ] n-
A useful feature of functional languages is partial application. For curried function, this adapts directly into math. But usually in math, for readability reasons, we use uncurried functions. For instance, let E, F and G be sets, u : E × F → G and (e, f ) ∈ E × F . We can write x → u(x, f ) and y → u(e, y), but this is a bit verbose. Thus, we also use a special symbol to be a placeholder: _. Any argument that receives this symbol is unbounded. For instance (x → u(x, f )) = u(_, f ) and (y → u(e, y)) = u(e, _). We also write _ i where i ∈ N * to reorder unbounded parameters. For instance ((y, x) → u(x, y)) = u(_ 2 , _ 1 ).

Notation

A.6 Standard Objects

We denote by π i the i th projection, that is the function that yields the i th component of a tuple: π i ((a 1 , . . . , a n )) := a i with i n. For all finite set E, we write S E the symmetric group of E, that is the set of all bijective functions of E → E. For all n ∈ N * , we write S n the symmetric group of degree n, that is S n := S 1,n .

For all set E, we write E the Kleene star of E, i.e. E is the set of finite sequences (that is, tuples) of elements of E:

E := ∞ i=0 E i
and we write

E + := ∞ i=1 E i
the set of finite non-empty sequences of elements of E. The Kleene star notation looks like notation of numerical set without 0, yet the star is different: N is the set of finite sequences of natural numbers, while N * is the set of positive natural numbers.

Given integers a, b and n, we write a mod n the binary modulo operation applied to a and n (that is, the remainder of the Euclidean division of a by n). We denote a ≡ b [n] the congruence relation between a and b modulo n, that is ∃k ∈ Z : a -b = kn Given a set E, we write Id E (or simply Id) the identity function: equivalent using ANSI escape code: we can apply the sequence to color the "a" in blue, then a sequence to color the "b" in red, but then, there is no sequence to cancel the last color; to color the "c" in blue, we have to apply the sequence for blue foreground again.

Id E : E → E x → x
That is, we must remember how it was before applying the red style. Most libraries using ANSI escape sequences simply reset the current style at the end. This prevents compositional use. For instance, in our example, "c" would get the default color (typically white on black terminals, and black on white terminals), because formatting "b" would reset the current style.

Thanks to Format's semantics tags, OColor keeps a history of previously applied types. It works by storing mutable stacks in the closure of tag management functions. There is one stack for each kind of style: a stack for the foreground color, one for the background color, one for the underline status (simply underlined, doubly underlined or not underlined at all). . . . Each stack, is a pair of a first class module and its state, this allows each stack to be handled differently. For colors stack, when the closing tag is encountered, the stack is popped and the previous color is applied. For styles that cannot be removed independently, like faint, we reset the whole state, and we reapply the style described by the top of each stack.

With OColor, we can print abc by writing Ocolor_format.printf "@{<blue>a@{<red>b@}c@}";

Many utility functions are provided as well as highly-configurable printers for common types such as Boolean, options, results, tuples, lists, arrays, iterable containers and iterable mappings. There are also some features to ease conditional pretty printing. Styles in semantic tags can be written in many ways, in particular, colors can be specified using 4-bit color names, RGB (red-green-blue) hexadecimal code, RGB decimal code, grayscale or X11 color name. We can also specify directly specific tags rather than names in the format string. We get the same result by writing Using OColor requires very few adaptations. Indeed, it is fully compatible with functions like Format.fprintf, it only needs to use formatters with semantic tag handling functions, thus, we only need to use OColor's version of printf, eprintf, asprintf and kasprintf. We may also use fprintf with formatters defined in OColor. With default formatters, semantic tags are simply ignored.

OColor does not detect the current terminal but its capabilities can be configured. For instance, many modern terminals handle 24-bit colors, but some of them can only work with 8-bit or 4-bit colors. In such a case, OColor will replace colors with the closest color possible in the most expressive encoding available. The choice of the closest color is made according to the Euclidean distance in CIE § L*a*b* color space § French: Commission internationale de l'éclairage, International Commission on Illumination B.2. EPICTETUS 383 (see [START_REF] Cie | 2019(E). International Commission on Illumination (Commission internationale de l'éclairage)[END_REF]) under CIE Standard Illuminant D 65 , as it is made to be faithful to the human perception of colors. This functionality is not only made to support outdated terminals, but also to adapt colors to the user's convenience. Indeed, while 24-bit colors are absolutely defined, 4-bit colors are just names, the real rendered color depend on the configuration of the terminal, allowing the user to choose its colors depending on the background. As a consequence, to find the closest 4-bit color, OColor should know terminal's configuration. It can be supplied manually, but several classical configurations (like the one used by gnometerminal in Ubuntu) are ready to use.

OColor is very useful in Astrée where log files are very long and need to be more readable, and printers are almost always defined inductively. OColor is perfectly fitted to this task because, as long as each printer close all the tags it opened, sub-printers will not interfere with the previous formatting.

OColor also have a version for Printf, but because of the absence of semantic tags, it has no memory of the past, like most libraries using ANSI escape codes.

B.2 Epictetus

Epictetus stands for "Elegant Printer of Insanely Complex Tables Expressing Trees with Uneven Shapes". It is an OCaml library that provides functions to print tables in which columns may have arbitrarily many and deep sub-columns. The structure of each line is not necessarily the same, yet we want columns corresponding to the same position in the tree of columns to be aligned. It has a public version available at [Che20a]. The name Epictetus also recall the Greek philosopher Ἐπίκτητος who, as a Stoic, would have liked perfectly aligned tables. Epictetus is about 500 loc long and has no run-time dependency.

It was originally designed to made assembly rules generated by Amical (see section 9.4 "A more Concise Syntax for Assembling Rules" (page 128)) more readable, to be easily compared with Intel's manual. Afterward, it has been used to visualize many kinds of data, e.g. timing results.

Epictetus is also written to work nicely with OColor, or any use of semantics tags. Indeed, Epictetus needs to compute the length of each cell to align nicely columns. But OColor (among other), generate non-printable characters using semantics tags. Such characters should not be taken into account when computing the length of the code ¶ . Epictetus provides several ways to deduce the apparent lengths of strings to print.

B.3 Plato

Plato is an adaptation of relevant parts of Python's standard library to OCaml. It has a public version available at [Che20c]. Its name stands for "Python Library Adapted ¶ Actually, we can note that Format ignores these characters with respect to formatting. To check if a test is successful or not, we cannot compare the outputs: there are very unstable in general, especially during development, where there are many debug printing. Moreover, even with a single version of Astrée, the output contains information on the environment (which are specific to each computer), execution time and starting date and time. In addition, alarms are not guaranteed to be issued in the same order, especially alarms about the same line of code. Logs are parsed to extract relevant data; the result is called summary. Then, we need to compare summaries meaningfully. For instance, the set of alarms should be the same, independently on the order. Summary comparison may also return non-fatal errors (i.e. warnings), e.g. when execution time of the test is very far from the standard (it may detect a major performance regression). To investigate errors more easily, standards are not reduced to the summary: we also keep the log file from which the standard was extracted. Also, this allows the user to regenerate summaries when we change the parser. Summaries are stored in JSON files, pointed by leaves of the DAG of tests. Each file contains a summary for each execution context, that is the number of summaries is at most the number of paths from the root † † .

In a legacy version of Socrates, configuration at each node was given as a list of command line options. This solution was deprecated since it has several limitations. Firstly, it is not always possible to override an option set before. Moreover, the analysis behavior does not depend directly on the command line options, but on the internal configuration of Astrée resulting from these options. In particular, it changes if the semantics of an option change ‡ ‡ or the default value of a parameter is modified § § . This † † It is not necessarily equal since different paths may result in the same configuration.

‡ ‡ This happens quite a lot during development. Typically, an option sets an integer parameter that has two uses. This parameter is split into two different ones, for each use, and command line options are updated accordingly.

§ § For instance, during development, many debug flags are set by default, but should be disabled to

  Figure 2.1: Hexadecimal numbers by 16-bit packs

Figure 2

 2 Figure 2.3: Alternate EAX register names

Figure 2 . 5 :

 25 Figure 2.5: Segmentation and paging, from [Int20]

Figure 2 . 6 :

 26 Figure 2.6: Real-address mode address translation

Figure 2 . 7 :

 27 Figure 2.7: Layout of a segment descriptor

2 :

 2 Dereferencing a pointer to stack or heap

Figure 2 . 8 :

 28 Figure 2.8: Structure of a segment selector

Figure 2 . 12 :

 212 Figure 2.10: CR3 register

4 :

 4 To get the value of EIP with guile

Listing 2 . 5 :

 25 Writing a far call using defines

Figure 2 . 13 :

 213 Figure 2.13: Structure of a 32-bit call gate

Figure 2 . 16 :

 216 Figure 2.14: Structure of a task gate

Definition 4 . 1 -

 41 Top-level definitions program ::= { toplevel } toplevel ::= function_definition | global_definition function_definition ::= type fun_name '('[ params ]')' params ::= type name ',' params | type name type ::= 'int8' | 'int16' | 'int32' | type '*' 44 CHAPTER 4. A MINIMAL C-LIKE LANGUAGE global_definition ::= type name ';'

Definition 4 . 2 -

 42 Programs, functions and bodies We denote P C the set of programs, and given P ∈ P C , F [P ] ⊆ F the set of function names used in the program P . Given P ∈ P C and f ∈ F [P ], Body P (f ) is the body of the function f in program P . Definition 4.3 -Statements block ::= '{'{ c_stat }'}' c_stat ::= block | type name ';' | lvalue '=' expr ';' | lvalue '=' fun_name '('[ expr_list ]')' ';' | 'if' '(' expr ')' c_stat | 'while' '(' expr ')' c_stat | 'return' expr ';' lvalue ::= name | '*' i expr expr_list ::= expr | expr ',' expr_list expr ::= '['a',' b']' | '&' lvalue | lvalue | op '(' expr { expr }')'

  n , in P ((S i , π.i)) ∩ in P ((S j , π.j)) = ∅ -otherwise in P (C) = {at P (C) , after P (C)} Notation 4.3 -Labeled statement Given a program P ∈ P C , a statement S and two labels (l, m) ∈ L C 2 , we write l S m :⇔ ∃π ∈ Π : at P ((S, π)) = l ∧ after P ((S, π)) = m

Notation 4. 5 Given

 5 (a, b) ∈ Q 2 and τ a transition relation over Q, a → τ b :⇔ (a, b) ∈ τ

  that maps register names to their first byte in A R , and we extend the function | • | to R to give as well the byte-size of registers:

7 }

 7 Listing 6.3: An illegal example involving too many memory operands

6 } 6 . 4 :

 664 Listing Using a C variable with register storage class in an assembly block

5 :

 5 Using a C static local variable in an assembly block

12 }

 12 Listing 6.6: Non-constant set of living local variables

8 :

 8 An assembly label as a function callable from C as the assembly label.

}

  int f() { int a, b, c; // ... C code ... asm { mov EAX, a[EBP]; add EAX, b[EBP]; mod c[EBP], EAX; } // Like c = a + b, but with safe integer overflow // ... C code ... 10 Listing 6.9: Adding with safe wrap-around

1

  extern void g(void);

  returnWe need to discover dynamically the role of each control flow instructions. This deduction is based on the destinations which can be computed from the mnemonics, arguments and memory states.

  3: A return to a new function

  [ESP] ; return address of n-th call mov d, EAX ; so address of (n+1)-th call } n = (d -(int)one) / ((int)one -(int)zero); asm { ; ^not zero, to get the address of the n-th call add ESPint n) { int d = (int)zero + n * ((int)one -(int)zero);

  .. same with other registers ... */ } regs; Listing 7.6: A naive C model of registers

Definition 8 . 2 -

 82 Statements and programsWe denote Stat the set of Panacea statements. We denote P the set of Panacea programs.

5 }

 5 Listing 8.1: Two instructions separated by a label

L

  asm := L C × N and L := L C L asm Likewise, we extend at P : Cmp (P ) → L after P : Cmp (P ) → L in P : Cmp (P ) → P (L)

7 ; 8 } 8 . 2 :

 7882 (l, 2) ∈ L asm Listing Explicitly labeled version of Listing 8.1

}

  The remaining component of the memory model in Asclepius makes the call stack: there are local environments and void f(void) { 42 ; Will set ZF status flag since EAX == 42 here } Listing 8.3: Two immediately successive assembly blocks

12 }

 12 void g(void) { asm { mov EAX, X[EBP] ; What should be X to retrieve 42 ? Listing 8.4: Accessing deep assembly call frames

8 }

 8 Listing 8.6: Array overflow

†

  Using keywords BYTE PTR (1 byte), WORD PTR (2 bytes), DWORD PTR (4 bytes), in Intel syntax. 121 122 CHAPTER 9. AMICAL mov DWORD PTR [EAX], 1 and this one mov BYTE PTR [EAX], 1

  Figure 9.1: Instruction format

  11.1. ORDERING AND LATTICES 143 Definition 11.11 -Contractive and extensive maps Let (E, ) be a poset and f

  We want to prove that∀(c, a) ∈ C × A, γ(a) c ⇐⇒ a α(c) Which is exactly the hypothesis C γ ← ----→ αA up to the symmetry of ⇔, simply using the definition of (resp. ) from (resp. ).Proposition 11.11 -Alternative definition of Galois connectionsLet (C, ) and (A, ) be posets, α : A → C and γ : C → A be maps. The four properties:

Theorem 11. 5 -

 5 Kleene fixpoint theorem Let E = (E, ) be a CPO, f : E → E and a ∈ E. If a f (a) and f is continuous, then lfp a f exists and lfp a f = E {f n (a) | n ∈ N} where f n is the n th iterate of f .

7 }

 7 Listing 12.1: When merging two instructions can improve precision 155

Figure 13 . 1 :

 131 Figure 13.1: Structure of the composite domain of Astrée

  Listing 14.1: Applying integer arithmetic to a pointer

}

  Listing 14.3: Copying through registers in pseudo-C

Figure 14 . 1 :

 141 Figure 14.1: New structure of the composite domain of Astrée

Figure 14 . 2 :

 142 Figure 14.2: Relation between signed and unsigned integer with same representation

14. 5 .

 5 ENVISAGED IMPROVEMENTS 181 the addition of a constant value never increases the size of the interval.

  0000

Figure 14 . 3 :

 143 Figure 14.3: Representation of wrapped intervals with n = 4 bits

2 { 7 }

 27 1: A rejected legal code int f() //+-----unknown prefix of the stack { //↓ +--suffix asm { ; ...| push 1 ; ...| 1 | pop EAX ; ...| } // Empty suffix: OK! } Listing 15.2: A provable legal code The suffix cannot be stored as independent variables since they are contiguous in memory, as illustrated by Listing 15.3: popping the stack is replaced by an addition 15.1. ABSTRACTING STACK IN A SINGLE ASSEMBLY BLOCK 185 1 int f() //+-----unknown prefix of the stack Listing 15.3: A slightly more complicated provable legal code into ESP. The suffix is stored in an array a of constant size N.

5 :

 5 Two PUSH statements in a loopAs the code may contains several loops, one can represent the whole stack as a sequence of modulo-smashed segments, thanks to segment smashing.Of course, these are only simple strategies that have been selected as they are satisfactory on simple examples. Better-fitted abstractions must be designed according to the code one would like to analyze, with the help of the extensive literature on array abstraction, e.g. [Čer03; HP08; CCL11].

Figure 15 Figure 15 . 2 :

 15152 Figure 15.2: State of the stack in h in the example of Listing 15.6

  size ::= 'BYTE' | 'WORD' | 'DWORD' signedness ::= 'SIGNED' | 'UNSIGNED' arith2 ::= '+' | '-' | '*' | '/' | '%' | '&' | '|' | '>>' | '<<' expr ::= 'Name'

  cmp_op ::= '==' | '!=' | '<' | '<=' | '>' | '>=' cond ::= 'Cmp' '(' expr ',' cmp_op ',' expr ')' | 'And' '(' { cond } ')' | 'Or' '(' { cond } ')'

  static_cond ::= 'Static_bool' '(' bool_name ')' | 'Static_and' '(' { static_cond } ')' | 'Static_or' '(' { static_cond } ')' | 'Static_not' '(' static_cond ')'

2 extern

 2 void l(void); // Necessary to take the address of l in C code.

  int*)(a + 1)) = offset; asm { jmp a ; Runs the instruction in a that jumps to l } } Listing 16.1: Simple dynamic code with problematic handling

8 ;

 8 Here EBX == 2 Listing 17.1: Computing a program point as a pointer

2 :

 2 Executing code reachable only through a label

Listing 17 . 4 :

 174 Forward and return distant jumps f. At the end of the function, the current property is ⊥, and we have saved the map h:(EAX → 1).

  int x[4]; int* p = &x[0]; p++; Listing 18.1: Simple operations on pointers

1 5 return rebuilt 6 }

 56 int f(int x) { 2 int low = x & 0xffff; 3 int high = x >> 16; 4 int rebuilt = low | (high << 16) Listing 18.2: A complicated way to do nothing

Figure 18

 18 Figure 18.1: A variable defined by bit slices

  d] (v) the ghost variable whose bits a to b are used to represent bits c to d of variable v, with the constraint b -a = d -c. In this nomenclature low is called Slice [0:15] [0:15] (low). Of course, there are several legal values for Slice

  d] (v): bits outside the segment a, b are not constrained. But the value of v at the moment where we created Slice

Figure 18 . 2 :

 182 Figure 18.2: A variable defined by bit slices with a more systematic terminology

  15] (high) and 16 lower bits are the 16 lower bits of Slice

  15] (low), as shown on Figure18.3. Thanks to another domain, like a simple * Or by anything that has a short lifetime.

Figure 18

 18 Figure 18.3: Rebuilding a sliced variable

  15] (high) = Slice

  15] (low), so we can simplify this expression as on Figure18.4. Then int rebuilt gets the value of Slice

Figure 18 . 4 :

 184 Figure 18.4: Rebuilt value

  15] (high) . Both these new offsets are equal to Offset (x), which is needed to prove that Slice [0:15] [0:15] (low) = Slice

  15] (high) and then, to prove that rebuilt = x.

}

  Listing 19.1: A single memory zone with multiple representations translated to the assignments x ← ABCDEF01H y 0 ← x [0 : 7]

Var

Axiom 19 . 4 -

 194 Variables read and written by Assign Assign ((v, e), d) writes only variable v: ∀v ∈ V, ∀e ∈ E, ∀d ∈ D, ({v} ∪ Var (e) ⊆ supp (d)) ⇒ d |supp(d)\{v} = Assign ((v, e), d) |supp(d)\{v} Assign ((v, e), d) reads only variables in Var (e):

Axiom 19 . 5 -

 195 Variables read and written by Guard Guard (c, d) enforces a condition but does not modify the state: ∀c ∈ C, ∀d ∈ C, Var (c) ⊆ supp (d) ⇒ Guard (c, d) ⊆ d Guard (c, d) only depends on value of variables in Var (c):

  15] (low) and Slice

  RolesLet R be a set of unary constructors. These constructors are called roles.For instance Offset and Slice

  d] (x) is the subset of V I where bits a to b of Slice

Figure 19 . 4 :

 194 Figure 19.4: A more complicated constraint DAG

  the source has a successor), -∃!s ∈ N : (∀(a, b) ∈ E, a = s) (there is exactly one sink), -∀s ∈ N, (∀(a, b) ∈ E, a = s) ⇒ (∃(a, b) ∈ E : s = b) (the sink has a predecessor), -u : N U associates constraint to nodes of the DAG.

n 0 n 1 Figure 19 . 5 :Notation 19 . 10 -

 11951910 Figure 19.5: A trivial constraint-DAG

1

  int* x = &t[idx]; // Given int t[] and int idx 2 int low = x & 0xffff; 3 int high = x >> 16; 4 int* y = low | (high << 16);

  d] that we have seen previously. Domain Offset needs only one role: Offset. The last domain, Equality is only in charge of remembering equivalence classes and thus does not need any ghost variable, and consequently owns no role.

  15] (y) = x Slice [16:31] [16:31] (y) = x

  n, D , γ, , lfp , Assign, Guard, Alloc, Kill, Extract, Refine, ⪡, U, B Assumption 20.1 -Soundness of

Assumption 20 . 4 -Assumption 20 . 5 - 20 . 6 -Assumption 20 . 7 -Assumption 20 . 8 -

 204205206207208 Soundness of Guard ∀c ∈ C , ∀a ∈ D , Guard (c, γ(a)) ⊆ γ(Guard (c, a)) Soundness of Alloc ∀v ∈ V , ∀a ∈ D , Alloc (v, γ(a)) ⊆ γ(Alloc (v, a)) 260CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT Assumption Soundness of Kill ∀v ∈ V , ∀a ∈ D , Kill (v, γ(a)) ⊆ γ(Kill (v, a)) In particular, Alloc and Kill have the same effect on the support as their concrete counterparts. Soundness of Extract ∀io ∈ IO , ∀a ∈ D , γ(a) ∩ γ IO (io) ⊆ γ IO (Extract (io, a)) Soundness of Refine ∀io ∈ IO , ∀a ∈ D , γ(a) ∩ γ IO (io) ⊆ γ (Refine (io, a))

Notation 20 . 1 -

 201 Support of an abstract value Let a ∈ D . We write supp (a) := supp (γ(a)) Assumption 20.9

  31] (y) ← Slice

  15] (y) ← Offset Slice [0:15] [0:15] (low) -Offset Slice

[

  16:31] [16:31] (y) ← Offset Slice

  15] (high)

Definition 20. 8

 8 Let us denote ⩿ the order relation on E defined by a ⩿ b :⇔ MVar (a) DM MVar (b)

  15] (low) variable whose 16 lower bits are the 16 lower bits of low. Thus, the assignment low := x & FFFFH is translated into the assignment Slice [0:15] [0:15] (low) := x. Since the same multiset of ghost variables is involved, the first order is not permissive enough. But x is definitely simpler than x & FFFFH as it is structurally included. That what takes into account. 1 int x; 2 int low = x & 0xffff; Listing 20.3: Extracting a slice Definition 20.10 -Depth of a variable

  int x, low; if(?) { low = x & 0xffff; // low = x & (2 16 -1) } else { low = x & 0xfffffff; // low = x & (2 28 -1) } Listing 20.4: A join with different support In Listing 20.4, the support always contains two real variables: x and low. After the third line, we need another ghost variable: Slice

15 ]

 15 15] (low) that stores the value of x, and Slice knows that (low) [0 : 15] low After line 5, the support contains the ghost variable Slice [0:27] [0:27] (low), which gets the value of x and the domain remembers that 27] (low) [0 : 27] low

  27] (low) and then delete the latter. Now, in both state, the support is made of real variables, Slice [0:15] [0:15] (low) and Slice

  27] (low). We can perform the union, and we get 15] (low) [0 : 15] low That is, after line 6, bits 0 to 15 of low are bits 0 to 15 of Slice [0:15]

Definition 20 . 12 -

 2012 Depth of an abstract state Let a ∈ D be an abstract state. We call depth of a the value depth (a) := max {depth (v) | v ∈ supp (a)}

276CHAPTER 21 .

 21 BASE-OFFSET POINTER DOMAIN int x[4]; int* p = &x[0]; // p = (x, 0) p--; // p = (x, -1 × 4) p += 5; // p = (x, 4 × 4) p--; // p = (x, 3 × 4) = &x[3] *p = 42;

5 } else { 6 p 7 }

 567 = &x[1]; // p = (x, 4) // p = ({NUM, x}, {4, 42}) Listing 21.2: A variable containing an integer and a pointer value

Simple counting allows us to find that this defines 32 i=2 i 2 = 6 -

 26 32(32+1)(2•32+1) 1 = 11439 distinct roles. Informally, the semantics of a ghost variable Slice

  b+s] v is the set of states where its bits a to a + s (both bounds included) are equal to bits b to b + s (both bounds included) of v. We excluded Slice [0:31] [0:31] because it would simply state that both variables are equal. b+s] (v) ∈ supp (s) Slice

7 ]

 7 (y) and Slice

  j] (v) should exist. Formally,

  d] (x), bits outside the segment a to b are useless. If bits from a to b are a ONE slice, we could simply say that bits c to d of x form a ONE-slice and not use a ghost variable. If they are a slice, we could transfer it to x as well. This is also true if bits a to b of x are made of several slices. Now we have abstract states, it is time to define the concretization, which is what allow us to decide what is sound. We have already given a good insight on the con-cretization. Let us state it formally

e

  ZERO ONE (a, b, u) ⦑~e⦒ ONE ZERO These binary operations are commutative. Not to overcharge the following tables, we omit the lower half. ⦑e & f ⦒ ZERO ONE (c, d, v) b, u) if (a, b) = (c, d) and u ≡ v otherwise

  ⦑e | f ⦒ ZERO ONE (c, d, v) ZERO ZERO ONE (c, d, v) b, u) if (a, b) = (c, d) and u ≡ v otherwise ⦑e ^f ⦒ ZERO ONE (c, d, v) ZERO ZERO ONE (c, d, v) b) = (c, d) and u ≡ v otherwise ⦑e f ⦒ ZERO ONE (c, d, v) b, u) if (a, b) = (c, d) and u ≡ v otherwise ⦑e f ⦒ ZERO ONE (c, d, v)

  Let us explain what to do with each component of a generic abstract domain, as shown in definition 20.3 "Generic abstract domain" (page 258).The domain name n ∈ N is arbitrary, but it is useless to give it any roles. Most components are unchanged:-D -γ : D → D -: D × D → D -lfp : (D → D ) → (D → D ) -Assign : A × D → D (natural extension of the assignment of real variables) -Guard : C × D → D (idem) -Alloc : V × D → D (idem) -Kill : V × D → D (idem)294 CHAPTER 23. OTHER DOMAINS -Extract : IO × D → IO (adapted as explained in section 20.2 "Product of Domains" (page 263)) -Refine : IO × D → D (idem) But ghost variable-related objects are yet to be defined:

Figure 24 . 1 :

 241 Figure 24.1: New structure of the composite pointer domain of Astrée

1 4 match Hashtbl.find_opt t a with 5 | 6 |

 456 let memoize (type a b) (f: a -> b) : a -> b = 2 let t = Hashtbl.create 0 in 3 fun a -> None -> let b = f a in Hashtbl.add t a b; b Some b -> b Listing 24.3: Mainstream memoization in OCaml

Figure 24 . 2 :

 242 Figure 24.2: An extreme usage of loggers

2 p 5 // ... 6 }

 256 = p & 0xfffffffc; // Enforcing alignment 3 // ... where p is not modified 4 if(!(p & 0x3)) { // Testing alignment Listing 25.1: Enforcing and testing alignment

xTFigure 27 FFigure 27 . 2 :F

 27272 Figure 27.1: Flat partitioning

∀

  (c, d, x), F alse d (F alse c (x)) → F alse c (F alse d (x))∀(c, d, x), T rue d (F alse c (x)) → F alse c (T rue d (x)) ∀(c, d, x), F alse d (T rue c (x)) → T rue c (F alse d (x)) ∀(c, d, x), T rue d (T rue c (x)) → T rue c (T rue d (x))which hold since each rule involves two variables that abstract the same concrete state, but we also need additional computations to create F alse d (x) and T rue d (x). Typically,F alse d (F alse c (x)) F alse d (T rue c (x)) → F alse d (x) T rue d (F alse c (x)) T rue d (T rue c (x)) → T rue d (x)

F

  Descriptor privilege level . . . . . . . . . 14, 19 DF . . . . . . . . . . . . . . . . . . . . . . see Status flag DPL . . . . . . see Descriptor privilege level DS . . . . . . . . . . . . . . . . see Segment register Dual poset . . . . . . . . . . . . . . . . . . . . . . . . . 137 Dynamic code . . . . . . . . . . . . . . . . . . . . . . 199 E EFLAGS . . . . . . . . . . . . . . . . . . . . . . . 11, 177 EIP . . . . . . . . . . . . . see Instruction pointer Elementary ghost constraint . . . . . . . . 248 Environment domain . . . . . . . . . . . . . . . 168 ES . . . . . . . . . . . . . . . . see Segment register Exception . . . . . . . . . . . . . . . . . . . . . . 32, 319 Extensive map . . . . . . . . . . . . . . . . . . . . . 143 Extensivity . . . . . . . . . . see Extensive map Extremum . . . . . . . . . . . . . . . . . . . . . . . . . 139 Far call. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30 Far pointer . . . . . . . . . see Logical address Fault . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Fixpoint . . . . . . . . . . . . . . . . . . . . . . . . . . . 143 Floating point unit . . . . . . . . . . . . . . . . . 202 FPU . . . . . . . . . . . . see Floating point unit FS . . . . . . . . . . . . . . . . see Segment register G Galois connection . . . . . . . . . . . . . . .

Figure A. 2 :

 2 Figure A.1: Layout of a 8-element array of 1-byte values

  [a : b] where s is the sequence, and a and b are respectively the indices of the beginning and the end of the extracted sequence. In 370 APPENDIX A. NOTATIONS AND BASIC DEFINITIONS the context of a graphical representation, we may put the end before, so that a b, to imitate the bit ordering. For instance, given a 32-bit integer n, Figure A.2 shows how to use this notation to switch between big-and little-endian representation. For example, if n is 12345678H (colored by packs of 8 bits) then n [7 : 0] is 78H, n [15 : 8] is 56H, n [23 : 16] is 34H and n [31 : 24] is 12H. After the transformation of Figure A.2, we would get 78563412H. Overall, we write indistinctly s [a : b] and s [b : a].

  is the new symbol. One could as well write unambiguously b =: a. In the same way, we write A :⇔ B and B ⇔: A to define A as equivalent to B. * French: Système international, International System of Units A.3. SET THEORY 371 We also use the Iverson's bracket notation. Given a proposition P [P ] = 1 if P 0 if ¬P This notation subsumes other notations like the indicator function (1 A (x) = [x ∈ A]) or the Kronecker delta (δ i,j = [i = j]).We denote tt (resp. ff ) the truth value "true" (resp. "false").NotationMeaning a := b defines a as equal to b b =: a defines a as equal to b A :⇔ B defines A as equivalent to B B ⇔: A defines A as equivalent to B [P ] Iverson's bracket tt logical true ff logical false

  [a, b], and the corresponding open interval is denoted ]a, b[. We do not use the style (a, b) because it is indistinguishable from the ordered pair. The order of bounds of a real interval does not matter since [a, b] := {t • a + (1 -t) • b | 0 t 1} So [0, 1] = [1, 0]. For integer intervals, we use the notation a, b . In this case, the order matters since a, b := {x ∈ Z | a x b}

  f |X = (X ∩ A, B, G ∩ (X × B)) and f |Y = (A, Y ∩ B, G ∩ (A × Y )).Of course, a function can be restricted and corestricted.Another interesting way to build function is to update a single image. Let f : A B, a ∈ A and b ∈ B. We denote f [a → b] the function

  the definition f [a i → b i ] n i=1 = (A, B, {(x, y) ∈ G | ∀i ∈ 1, n , x = a i } ∪ {(a i , b i ) | i ∈ 1, n })

B. 4 .

 4 Figure B.1: A widget to duplicate tests

  . They are set by CHAPTER 2. THE X86 ARCHITECTURE AND SYSTEM PROGRAMMING

	31	30	29	28	27	26	25	24	23	22	21	20	19	18	17	16	15	14	13	12	11	10	9	8	7	6	5	4	3	2	1	0

8 • e [8 : 10] + 2 12 • e [12 : 31]

  Here a == 1 and b == 2

	5	}
	6
	7	int f() {
	8	int x;
	9	asm {
		push ECX	; We might want to save it,
			; since it is a caller-saved register,
			; even if we do not use it here.
		push 2	; Second parameter
		push 1	; First paramter
		call g	; Call C function

3 // ... C code ... 4 return 4; mov x[EBP], EAX ; Here EAX == 4 add ESP, 8 ; Stack clean-up pop ECX } return x; // Returns 4 } Listing 6.11: Assembly-to-C mixed call

Table 10

 10

	.1: Rule of signs for multiplication
	With another operation, the partial information may be insufficient. Let us consider

Table 10

 10

	.2: Rule of signs for addition
		?
	+	0	-
		⊥
	Figure 10.1: Sign lattice

  12.1. EXAMPLE: WHEN REDUCTION IS NEEDED157 values in the interval are lower than 10 thus everything enters the loop. After another iteration, we get 1, 5 , and so on. After 5 iterations, we have 1, 11 . Something is different here: only 1, 10 enters the loop again, and 11, 11 exits the loop. Since the previous state was 1, 9 , we do not have a fixpoint yet: 10 is a new possible value. It is interesting to remark that 10 is not possible in the concrete execution since v is always odd. We perform another iteration, and we end with 1, 12 at the top of the loop. Here 1, 10 enters again and 11, 12 exits. We reached a fixpoint. The Table12.1 sums up the abstract properties at each iteration. In green, we show the inductive invariant that holds at the beginning of loop's body.

	Number of iterations	0	1	2	3	4	5	6
	Interval at the top of the loop	1, 1	1, 3	1, 5	1, 7	1, 9	1, 11	1, 12
	Interval staying in the loop	1, 1	1, 3	1, 5	1, 7	1, 9	1, 10	1, 10
	Interval exiting the loop	∅	∅	∅	∅	∅	11, 11	11, 12

Table 12

 12

.1: Abstract properties at each iteration Before the line 6, possible values, according to interval domain are 11, 12 .

  Pointer domain. It builds a domain that is able to abstract pointers and numerical variables using a domain that abstract numerical values. It takes the composite relational numerical domains in parameter, and uses it to represent directly numerical values or the offset part of pointers. This domain is explained in detail in[START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF][START_REF] Miné | Field-sensitive value analysis of embedded C programs with union types and pointer arithmetics[END_REF]. This domain is not able to handle dereference: it knows what pointers point to, but at its abstraction level, variables are memory cells and expressions do not longer contain indirections: they are solved by the immediately upper domain: struct domain. 2. Struct domain. This domain abstracts C aggregate types (structures, union and array). Also, it resolves pointers: above this domain expressions can use C variables and dereferences; under it (like in pointer domain), expressions contain only physical cells, and indirections are solved and thus absent from expressions. To resolve dereferences, it uses points-to information given by the pointer domain. This domain is also exposed in[START_REF] Miné | Static analysis by abstract interpretation of concurrent programs[END_REF][START_REF] Miné | Field-sensitive value analysis of embedded C programs with union types and pointer arithmetics[END_REF].

1.

  parameters are passed on the stack in right-to-left order; -result (if any) is returned via EAX; -EBX, EBP, ESP, ESI and EDI are callee-saved, other are caller-saved.

	1	extern int h(int);
	2
	3	void h_wrapper() {
	4	asm {
	5	h: ; We do not need to save callee-saved
	6	; registers as we do not modify them.
	7	mov EAX, [ESP + 4] ; copy the parameter in EAX, thus EAX <-1
	8	add EAX, 1337
	9	ret	; Return EAX (1338)
		}
		}
		int g(int a) { // here a == 42
		return a + h(1); // return 42 + 1338 (== 1380)
		}
		void f() {
		int result;
		asm {
		push 42	; Pushes the argument of g on the stack
		call g	; (We do not need to save
			; caller-saved registers)
		add ESP, 4	; Cleans the stack
		mov result[EBP], EAX ; copy the returned value in result
		} // result == 1380

}

Listing 15.6: An alternation of C and assembly call frames

Table 16

 16

	1	in EAX, 42	; Input doubleword from I/O port 42 into EAX
	2	add EAX, 1
	3	out 43, EAX ; Output EAX to I/O port 43
			Listing 16.2: Simple communicating program
			Input/Output Port Value
			Input	42	5
			Output	43	6

.1: Simple communication scenario

  4 "Termination" (page 266), we just use the same here. In this case, this relation will be just as restrictive as the one in other domains. -We allow this relation not to be well-founded for domains that does not use ghost variables. Since such a domain does not generate ghost constraints, there is no termination problem. Moreover, other domains are still subjected to a well-founded relation, as long as there is at least one domain that use ghost variables. If there is no such domain, termination is ensured because no domain generates constraints. With this formalism, we choose ⪡= U × U as it accepts everything. -

  Finally, the key dispenser knows two kinds roles: real and virtual ¶ . Ghost variables have a role of the form Virtual r where r is a value of the role defined inductively from domains. Roles of real variables have the form Real k where k is the outer key. It agrees with the notion of role (what the variable is useful for): a real variable represents the value of the variable identified by the outer key k.

	1	module type ROLE =
	2	(sig
	3	type t
	4	val compare: t -> t -> int
	5	val pp: Format.formatter -> t -> unit
	6	val get_keys: t -> int list
	7	end)
		Listing 24.2: ROLE signature
	underlying domains' module Role.

  32] × 2 12 + 4 × a [12 : 21] [12 : 31] = a [12 : 31] where CR3 is the page directory base register (see subsubsection 2.3.3.1 "Translation Mechanism" (page 25)). Of course, we need an extra property: accessed entries are marked as present (P flag is set). That is, the a [22 : 31] th entry of the page directory points to a page table whose a [12 : 21] th entry points the page containing a.

  . . 145 GDT . . . . . . . see Global descriptor table GDTR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 General-purpose register . . . . . . . . 10, 175 Generic abstract domain . . . . . . . . . . . 258 Ghost variable. . . . . . . . . . . . . . . . .229, 245 Ghostly ordering . . . . . . . . . . . . . . . . . . . 245 Global descriptor table . 19, 23, 196, 231 GP register see General-purpose register GPR . . . . . . see General-purpose register Greatest lower bound . . . . . . . . . . . . . . 140 Tarski's theorem . . . . . . . . . . . . . . . . . . 144 Task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Task gate . . . . . . . . . . . . . . . . . . . . 24, 33, 35 Task register . . . . . . . . . . . . . . . . . . . 35, 196 Task-state segment . . . . . . . . . . . . . . 24, 34 TI . . . . . . . . . . . . . . . . . . see Table indicator TLB . . . see Translation lookaside buffer Total order . . . . . . . . . . . . . . . . . . . . . . . . . 138 TR . . . . . . . . . . . . . . . . . . . see Task register Trace domain . . . . . . . . . . . . . . . . . . . . . . 168 Transition system . . . . . . . . . . . . . . . . . . . 49 Translation lookaside buffer . . . . . . . . . 28 Trap . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Trap gate . . . . . . . . . . . . . . . . . . . . . . . . 24, 33 TSS . . . . . . . . . . . . see Task-state segment U Upper bound . . . . . . . . . . . . . . . . . . . . . . . 140 V Virtual-8086 mode . . . . . . . . . . . . . . . . . . 14 W Widening . . . . . . . . . . . . . . . . . . . . . . . . . . 151 Z ZF . . . . . . . . . . . . . . . . . . . . . . see Status flag

	Appendix A

  .1. In practice, we use only the 3 lower ones.

	Value	Symbol Name
	2 10 1024	Ki	kibi
	2 20 1024 2 Mi	mebi
	2 30 1024 3 Gi	gibi
	2 40 1024 4 Ti	tebi
	2 50 1024 5 Pi	pebi
	2 60 1024 6 Ei	exbi
	2 70 1024 7 Zi	zebi
	2 80 1024 8 Yi	yobi
	Table A.1: Binary prefixes

Table A

 A A \ {0} (where A is a numerical set) a, b the set of integers between a and b A ↑R set of elements greater than an element of A according to R A ↓R set of elements lower than an element of A according to R Table A.3: Set theory notations Classical notations are recalled on Table A.3.

		.2: Logic notations
	A.3 Set Theory
	Notation	Meaning
	x ∈ A	x belongs to A
	A ⊆ B	A is a subset of B
	A B	A is a proper subset of B
	P (A)	the power set of A
	A ∪ B	union of A and B
	A ∩ B	intersection of A and B
	A \ B	A without B
	∅	the empty set
	N	set of natural integers (including 0)
	Z	set of integers
	N *	set of positive integers (so excluding 0)
	A

*

  Meaning B A set of functions from A to B f : A → B f is a function from A to B f : A B f is a partial function from A to B

	supp (f )	support of f
	Im (f )	image of f
	f |X	restriction of f to X
	f |Y	corestriction of f to Y
	f [a → b]	f except that a maps to b
		Table A.5: Logic notations

Table A .

 A 6 -Notations of Part I

	Notation	Defined Meaning
	⦉l⦊		def. 4.9 p. 51 semantics of lvalue l
	(e)		def. 4.9 p. 51 semantics of expression e
	V [l]		not. 4.7 p. 51 set of living variables at label l (program is
			implicit)
	M [V ]	not. 4.9 p. 52 set of Asclepius memory states where set
			of variables is V
	Ω		p. 52 set of possible errors
	τ P C	a S b	p. 52 set of transition generated from statement a S b in program P
	ω o		p. 54 out of memory error
	A R		p. 66 set of register byte addresses
	R		p. 67 set of register names
	H asm	p. 67 set of assembly heaps
	reg (r)	p. 67 address of first byte of register r
	E asm	p. 67 set of assembly environments
	Stat asm	p. 68 set of assembly instructions
	disassemble (h)	p. 68 disassemble next instruction to run in mem-
			ory state h
	ω i		p. 68 illegal instruction error
	M asm	p. 69 set of memory states in the most concrete
			interpretation of assembly
	M asm	p. 79 set of assembly memory states in the more
			symbolic semantics
	Stat	def. 8.2 p. 100 set of Panacea statements
	P		def. 8.2 p. 100 set of Panacea programs
	L asm	p. 101 set of assembly labels
	L		p. 101 set of Panacea labels
	E		p. 104 set of Panacea environments
	M		p. 104 set of Panacea memory states
	A.7

.2 Notations of Part II "Abstract Interpretation and the Astrée Analyzer"

  /n p. 238 constructor C of arity n E V not. 19.1 p. 239 set of expressions with variables in V E not. 19.2 p. 239 set of real expressions ⫑ def. 19.1 p. 239 structural inclusion of expressions O not. 19.4 p. 240 set of condition constructors C V not. 19.4 p. 240 set of conditions with variables in V C not. 19.4 p. 240 set of real conditions Alloc (v, d) p. 241 allocates variable v in state d Kill (v, d) p. 241 kills variable v in state d Assign (a, d) p. 241 performs assignment a in state d Guard (g, d) p. 241 enforces guard g in state d D p. 241 set of memory states sets with consistent support A V not. 19.7 p. 241 set of assignments with variables in V A not. 19.7 p. 241 set of real assignments R def. 19.2 p. 245 set of roles V i not. 19.8 p. 245 set of ghost variables at level i V not. 19.8 p. 245 set of ghost variables E not. 19.9 p. 245 set of ghost expressions C not. 19.9 p. 245 set of ghost conditions A not. 19.9 p. 245 set of ghost assignments (or directed constraints) def. 19.3 p. 245 ghost ordering ❲v❳ def. 19.4 p. 246 semantics of ghost variable v U V def. 19.5 p. 248 set of constraints with variables in V not. 19.11 p. 251 sink of constraint-DAG g ⟪g⟫ n def. 19.7 p. 251 semantics of node n in constraint-DAG g ⟪g⟫ def. 19.7 p. 251 semantics of constraint-DAG g N def. 20.1 p. 254 set of domain names o (r) def. 20.2 p. 254 owner of role r def. 20.2 p. 254 role and variable belonging relation Alloc (v, d) def. 20.3 p. 258 allocates variable v in abstract state d Kill (v, d) def. 20.3 p. 258 kills variable v in abstract state d Assign (a, d) def. 20.3 p. 258 performs assignment a in abstract state d Guard (g, d) def. 20.3 p. 258 enforces guard g in abstract state d n def. 20.4 p. 260 ghost constraint triggering relation in domain named n ⟪g⟫ def. 20.5 p. 262 abstract semantics of constraint-DAG g ⦉(new, old, g)⦊ def. 20.6 p. 262 allocates variables in new, enforces g and kills variables in old MVar (e) not. 20.2 p. 267 multiset of variables in e ⩿ def. 20.8 p. 268 order on expressions used in def. 20.9 p. 269 order on constraints used in depth (v) def. 20.10 p. 269 depth of variable v def. 20.11 p. 270 termination relation depth (d) def. 20.12 p. 274 depth of state d H p. 282 set of possible values of a slice of a variable

		382	APPENDIX B. BY-PRODUCTS
			Table A.7 -Notations of Part II Table A.8 -Notations of Part IV
	Notation (C, ⊆) γ ← ----→ α Notation (A, ) not. 11.8 p. 145 Galois connection Defined Meaning Defined Meaning ⟜ g not. 19.11 p. 251 source of constraint-DAG g
	A.7.3 Notations of Part IV "A Reduced Product with Ghost Vari-
		ables"
			Table A.8: Notations of Part IV
	Notation	Defined Meaning
	V		p. 237 set of real variables
	I		p. 237 set of possible values of variables
	S		p. 237 set of memory states
	h		p. 238 set of expression constructors
	G C U u	Notation fp f lfp a f lfp f gfp a f gfp f Y p ⦑e⦒ H' Y' p' p ⧘ e| |e ⧙ ⧘ e ⧙ ⦕e⦖	Table A.7: Notations of Part II Defined Meaning not. 11.7 p. 143 set of fixpoints of f not. 11.7 p. 143 least fixpoint of f greater than a not. 11.7 p. 143 least fixpoint of f -Continued on following page --Continued on following page -not. 11.7 p. 143 greatest fixpoint of f not. 19.10 p. 251 set of constraint-DAGs not. 11.7 p. 143 greatest fixpoint of f lower than a p. 283 set of possible variable slices p. 283 set of sliced variables p. 285 state of a slice of sliced expression e p. 287 set of possible values of a slice of an expres-sion p. 287 set of possible expression slices p. 287 set of sliced expressions p. 287 set of modified sliced expressions p. 287 cut on the left the modified sliced expression e p. 288 cut on the right the modified sliced expres-sion e p. 288 cut both sides of the modified sliced expres-sion e def. 19.5 p. 248 set of ghost constraints p. 249 semantics of a ghost constraint p. 289 compressed sliced expression e

⊸

g

† For instance (*x = *x) | (*y = *y) is an undefined behavior if x and y are aliases. From such expression, the compiler may safely assume that x and y point to disjoint memory blocks.

* There are in fact some degrees of liberty in the x86 architecture left to the implementation (i.e. the actual hardware). This implementation-defined behavior are quite rare, and should rarely be considered as C-style implementation-defined behavior, and are rather the result of history. First, we cannot expect the manufacturer to document it, or the processor to stick with a single behavior. Moreover, unlike a compiler that is useful only when the developer compiles the program (only once in a known environment), the processor is used at each run, and is outside the scope of the developer since it is on the user's computer. Thus, except for very specific cases, one should avoid these implementation-defined behaviors.† Indeed, the semantics is often specified using natural language and might not be univocal. Even in pseudo-code, some elementary operations are written in natural language, especially those that are too complex to be written formally. Unfortunately, they are often also the most ambiguous statements.

* * In fact, we can find all possible byte sequences and read them starting with some offset. But we would get a titanic number of possibilities, involving very different kinds of instructions, so it is useless to try to handle such behavior that is buggy in legitimate code. Nevertheless, this method is used on purpose in security attacks.

† † A typical example is the Pentium F00F bug: the illegal instruction lock cmpxchg8b EAX (encoded as F0 0F C7 C8) freezes some Pentium processors rather than raising an exception[START_REF]Pentium ® Processor Specification Update[END_REF][START_REF] Collins | The Pentium F00F Bug. Dr. Dobb's[END_REF].

† That is, a local variable, which is automatically deleted at the end of the block.

† That is, they have the same types, but may have different names.

† In section 6.1 "Syntax of Assembly Blocks" (page 81), we mentioned that, in a syntactic extension, assembly functions can be declared as top-level items. Since there is no C function wrapping them, there is no way to declare C local variables. Excluding local variable is not a strong hypothesis: it comes naturally when assembly blocks are wrapped in dummy C functions.

¶ It can be a full compilation in the usual meaning, or a mere simplification to remove redundant constructs of C.

* * Up to 23 for rules implemented in Amical. † † Or at least, not true without more constraints. ‡ ‡ Given a set of such constraints, some assemblers may be optimal, though it would be wise to stay skeptical.§ § Assembler usually have very limited static analysis abilities, making them usually unable to decide whether a suboptimal encoding is indeed required.

¶ ¶ More precisely, the Minkowski sum.

¶ Frama-C does the exact opposite in CIL (C Intermediate Language) (see[START_REF] Necula | CIL: Intermediate Language and Tools for Analysis and Transformation of C Programs[END_REF]): it removes all redundant syntax features.

* * A GNU extension, called "Labels as Values", allows pointers to labels. Function pointers are also dynamic but it is more a problem of function calls, since they adapt poorly in CFGs.

* For instance, recent processors such as the Intel Core i9-10980HK have a cache of 16 MB while it allows up to 128 GB of memory.

† If we assume that the stack suffix must be empty at the beginning of each assembly block. If we implement the bridging described in subsubsection 8.3.3.3 "Bridging Blocks" (page 109), we should adapt slightly this heuristic.

‡ We do not really need the whole enriched heap, but it makes the formalization lighter and mathematically, copying huge structure is free. Here, since we are closer to the implementation, we are interested into saving exactly what is necessary.§ But, probably, deterministically from a lower level point of view.

† Indeed, because of imprecision, the default case of an exhaustive switch may not be ⊥. ‡ It should not be used to raise an alarm if the abstract state does not satisfy a property. We use Fail or Assert for that purpose. Absurd is used when a program point in the microcode is not reachable.

* At least in x86. In some architectures, especially RISC (reduced instruction set computer), instructions have a constant size and are aligned.

* The new implementation might be slightly less efficient. See section 25.3 "Performance Evaluation" (page 320) for a more detailed investigation about execution time.† With the previous notation, we could have written OffsetNUM, but it may look weird since the numerical value is not an offset.

* We recall that we use Linear B characters for short-lived variables (see page 238). The character H stands for the syllable "po", like "possibilities", since it stands for the possible values of a slice.

† Character Y stands for the syllable "si", which is the closest of "slice" available. ‡ Symbol p is the syllable "pa", for "partition", and because its shape looks like a sliced vertical line.

† This is an estimation because it is difficult to distinguish changes directly related to the new framework from other changes. Indeed, this new framework is closely connected to the support of inline assembly. Moreover, it is difficult to take changes in internal libraries into account, as they were motivated by the new framework, but also globally useful.

‡ For instance, communication channels: the deduced constraint about the postcondition is the constraint provided for the precondition of the next instruction.§ It only takes a command line flag to switch between the two frameworks.

¶ This was the first name of ghost variables, before realizing the concept of virtual variables if in fact the same as the well-known notion of ghost variables.

* * And the base-offset domain enforced the constraint b while the computation of a was suspended. Domains have the responsibility to be able to enforce ghost constraints (using ghostlier variables) when they suspend their execution. A simple strategy is to delay the modification until the end, by saving useful values in the context. With this strategy, the suspended computation is in fact applied after ghost constraints. Partially applying a may lead to an inconsistent state, but it might not be a problem to enforce ghost constraints, depending on the domain.† † Nevertheless, the order might change the execution time and the precision.

130

CHAPTER 9. AMICAL

Part I

Semantics of Mixed C and x86 Assembly

Part II

Abstract Interpretation and the Astrée Analyzer

Part III

Abstraction of Mixed C and x86 Assembly

for label r:. Current property after line 19 becomes EAX → 3, and we reach the end of the function this way. As there is no local jump map, this function is stabilized, and since the forward jump map is the same, all called functions are also stabilized. A Reduced Product with Ghost Variables CHAPTER 19. CONCRETE FRAMEWORK may look like a mere implementation concern, but reasons that allow using directed constraints are quite fundamental and should be shown in the formalization.

Definition 19.5 -Elementary ghost constraints

Let V be a set of variables. We denote

and, in particular, U := U V the set of elementary ghost constraints.

Letter U is due to the unary nature of these constraints: when enforced, they take a single precondition as parameter and return a single postcondition. Constraints that domains communicate are not elementary: it would not have been expressive enough. For instance, let us assume we have two (real) 32-bit variables x and y that have been built using 16-bit slices. This is illustrated on Figure 19.1, where a, b, c and d are arbitrary known integers. The guard x = y can be translated as Slice (y) >> d & 2 16 -1 This is a conjunction of elementary ghost constraints. This can be run by guarding by this both conditions sequentially. If we write c : D → D the semantics of a constraint c, to enforce x = y in the state s ∈ D, we need to compute f • g(s) where f = Slice Thus compound (non-elementary) ghost constraints should be able to express both sequencing and branching, while still being able to run it easily without worrying about termination. For these reasons, we choose DAGs as compound ghost constraints. In fact, the exact form of the intermediate language does not matter. If needed, it could include other terminating constructs like constant-bounded for-style loops. We choose this language made of DAGs because it is both simple and expressive enough.

Slice

Offset Slice [0:15] (high) are equal. Since they point to the same block, this also means that these pointers are equal. From this, we can deduce that bits 0 to 15 of low are equal to bits 0 to 15 of x, which is what we expect from the source code. But for the sake of the example, we will assume that this reduction did not occur, i.e. we only know that offsets are equal, not whole pointers.

At this point, we have the following support, made of 3 trees, one for each real variable:

x low high

Offset (x) Slice

There are three real variables and five ghost variables: three of them belong to Offset and two belong to Slice. We can remark that a previously mentioned assumptions is satisfied: when a variable exists, all less ghostly variables are alive, too.

256CHAPTER 20. GENERIC ABSTRACT DOMAINS AND REDUCED PRODUCT

Now, let us run the last statement: int* y = low | (high << 16);. Offset and Equality domains get nothing interesting from that. However, the Slice domain is more clever with bitwise operations. It can deduce that bits 0 to 15 of y are bits 0 to 15 of low, which are also bits 0 to 15 of Slice T erm 2 (r), but here, the former contains the value of q, while the latter contains the value of &x[3].

After line 10, we need to perform a join. Supports are already unified, but a direct join would be very imprecise. Let us sum up in a table:

But if variables are renamed in one branch, we would obtain a much nicer state, for instance, if we rename variables in the else branch:

In this case, the join is much more precise. After the union, r is the sum of two ghost variables: T erm 1 (r) that points to x with byte-offset 4 or 12, and T erm 2 (r) that stores the value of p (which is q). Line 12 can be precisely abstracted using a simple equality domain and noticing that T erm 2 (r) and p are equal.

In such a domain where ghost variables have a symmetrical meaning, abstracting precisely binary operations may require renaming ghost variables. This matter will be developed in section 27.2 "Ghost Variable Renaming" (page 333).

Chapter 24

Implementation

T he implementation of the proposed framework is not a direct conversion into code of the formal description. For maintainability, debugging and performance reasons, the way it operates is very different from the mathematical point of view.

In this chapter, we explain how the framework is implemented, what are the differences between the mathematics and the implementation, and it explains why the implementation was designed this way.

The question of the results obtained by this implementation will be exposed in chapter 25 "Discussion" (page 315); in particular, execution time will be discussed in section 25.3 "Performance Evaluation" (page 320).

Overview

This new product of domains with shared ghost variables has been implemented in Astrée, thus, using OCaml language [Ler+20]. The programming language is very important as it guides architecture choices. They would have been very different in C ++ or Python, for instance. In particular, in OCaml, it is common to make extensive use of modules and functors * . The implementation of this new framework implied an estimated † addition of 30 kloc and the deletion of 14 kloc, among the 200 kloc of Astrée. Moreover, it resulted into external libraries and tools for a total of 17 kloc of OCaml and 11 kloc of Python. Overall, integrating the new framework can be easily qualified as a major change in Astrée.

The general architecture is shown on Figure 24.1, where each node is a module. Module key dispenser belongs to translator, and translator belongs to the driver; they are presented on the side since they are not abstract domains. Their purpose will be explained later (respectively in section 24.2 "Roles" (page 299) and section 24.3 "Trans- values of driver's type t. This is also why roles are useful in the implementation: as they encode the semantics of the ghost variable, two variables with the same role get the same key. Roles in the code can be less precise than the one in the formalization. Indeed, we are only interested in identifying variables for binary operations. For instance, we defined the role Slice [c:d] to express that bits a to b of the ghost variable are equal to bits c to d of the overlaying variable. Here, we do not need to know which slice of the ghost variable is useful, we only need to know which slice of the overlaying variable is represented. If bounds (a, b) are different between the two states, it is the responsibility of the unification process (see section 20.5 "Binary Operations" (page 271)) to act accordingly. During the unification process, ghost variables representing a slice can be shifted so that the same segment is used; we can also give up and default to (as the theoretical framework does).

This has another consequence: ghost variables may not have roles. This is unadvised since a new key will be generated each time, and the unification is more complicated since two variables without roles (so with different keys) may need to be unified: that is we allocate a third key, and we rename both old values into this new key.

Key dispenser state is purely incremental: it starts with an empty state. When a new key is requested, there are two cases:

-if a role is provided, the key dispenser looks for a key that would have been already provided for this role and returns it if so, otherwise it generates a new key and add the correspondence between the role and the key to its registry; -if no role is provided, a new key is returned and will never be returned again. Roles are defined inductively from domains: each domain provides a module Role implementing a common module type ROLE signature, shown in Listing 24.2. Function val get_keys: t -> int list returns the list of keys of overlaying (more real) variables. Unlike the implementation, a role is not always a unary constructor. In particular, an example of zeroary role will be given later. Roles of arbitrary arity have been also dreamed of, but not used for now. There is a subtle terminology point: in the formalization, roles where unary constructors, here, the role is the constructor and overlaying variable.

Combinator functors define the type t of their module Role a sum type of type t of

TRANSLATOR CHAPTER 24. IMPLEMENTATION

Thanks to its predominant situation, the driver may also check dynamically that theoretical constraints that are still applicable in the implementation, indeed hold. This is useful to detect errors as early as possible, rather than pursuing the execution with unexpected data, and resulting in a late error that may be nightmarish to investigate. Though, these checks are very time-consuming (see section 25.3 "Performance Evaluation" (page 320)), and should be disabled for non-development usage.

Abstract Order

To reach an inductive invariant, we need to stabilize loops. We threw the modest veil of lfp on this problem. But, in practice, lfp is not elementary: we need to build it from simpler operators. Usually, we need:

-widening, to ensure termination; -union, intersection and narrowing, to improve precision; -a sound approximation of the concrete order (usually, the inclusion), to know when we reach a correct approximation of a concrete post fixpoint. The two first points are all binary operators, and the way we handle them has been explained in section 20.5 "Binary Operations" (page 271). But checking the inclusion is a slightly different problem. We have to return a truth value, tt or ff , not compute a new abstract state. The common point is that states we compare may have unequal support.

Though we need to perform support unification, we do not need to use a sound approximation of the identity. We simply need to reduce the problem to an equivalent (and hopefully simpler) problem. From states a and b, we generate respectively states a and b such that a b ⇔ a b. The idea is that there are two steps to check the inclusion: -abstract states must have the same layout; -values of ghost variables must be included. For instance, let us consider the base-offset domain. Let a and b, two abstract states and x a variable. To test whether a b, we need to test the inclusion between the two values of x in a and b. So, we first need to compare the sets of pointed variables (bases). If the set of variables pointed by x in a is not included in the set of variables pointed by x in b, then a b. This is an internal operation: we do not need to compare ghost variables, the single domain can already decide that the inclusion does not hold. If the set of bases in a is indeed included in the set of bases in b, we need to compare the offsets: the possible offsets in a should be a subset of possible offsets in b. And thus, we need to ask every domain to compare these ghost variables.

In practice, we proceed in these exact two steps. Firstly, we collect a set of variables to compare, layer by layer. It is initialized by with all real variables (i.e. variables of V 0 ), then we ask for the set of ghost variables the domains need to compare, and we get a subset of V 1 . We iterate until we reach the depth of the support. During this process, a domain can already claims that the state inclusion does not hold, and shortcut the process. Once we get the set of all ghost variables we need to compare, it is provided 24.6. DOMAINS

Chapter 25

Discussion

A bstraction quality depends on two main criteria: the precision and the perfor- mance. Indeed, there are no ultimate abstraction, they all have limitation, thus we can only judge if it is good enough to treat families of problems we are interested in (or we might consider in the future). Moreover, if the analysis is too slow, it might be conceptually interesting, but, in the end, it is useless as it cannot be used to prove properties on software. Without performance constraints, abstract domains are much easier to design.

We will explore these two points here: first the precision ("Which analyses can be successfully performed?"), then the question of performance ("How patient do I need to be?").

Successful Analyses

We recall we are interesting at proving safety properties. As usual, we want to prove the absence of RTEs. Moreover, our more original goal is to prove security property of a host platform, like memory isolation. We argued that these properties could be overapproximated by safety properties about low-level memory protection features. Moreover, in practice, the way these features are used is such that these approximate safety properties are quite satisfactory.

In particular, we were interested in proofs of the correctness of segmentation implementation (see subsubsection 2.3.2.2 "Protected Mode Segmentation" (page 19)). This is the original motivation for the slices domain.

Segmentation

We recall that segmentation setup implies cutting pointers using bitwise instructions to fill a segment descriptor (see subsubsubsection 2.3.2.2.1 "Segment Descriptor" (page 19)). Then, when accessing the descriptor, the processor internally rebuild the original pointer. Since we want to prove properties on segment we use, we need to be able to cut and Chapter 26

Proving Low-Level Properties

A nalyzing mixed C and assembly allowed us to prove some properties on segmen- tation, but otherwise, we only proved the absence of RTEs in analyzed sections. Handling mixed code is already quite a challenge, and we did not have time to prove other low-level properties.

Here, we will see two points that we need to handle properly for our target application. The first is paging: proving that paging is correct would be a huge step to memory protection. The second is a hack more specific to our application, though we believe it might be useful in many other codebases: it is accesses to a deeper call frame.

Paging

Paging is explained in subsection 2.3.3 "Paging" (page 25). It is an important feature of memory management. Unlike segmentation, paging is a very dynamic and shared between all processes.

In real world strategies of segmentation, the problem could be simplified since we use segments starting at 0 and spanning over the whole space (or almost). The address translation is thus the identity, which can be safely ignored: we just check the set up segments imply a trivial translation, then there is no need to distinguish the logical address space and the linear address space.

Paging does not use identity mapping in general, but when it does, this is for a very good reason (see subsubsection 2.3.3.3 "Identity Paging" (page 27)). Thus, we need to distinguish the linear address space and the physical address space. In fact, most variables are only in one of both spaces: variables used only before activation of paging can be represented only in physical address space. Variables used only after activation of paging can be represented only in linear address space. There is a problem for variables that are alive during the activation of paging.

Let us break down this problem into more elementary ones. First, we shall prove that variables alive when paging is enabled are in a portion of memory that is identity mapped. This is not a formal requirement of the processor, but this is the usual solution since it might be awfully complicated otherwise. To do so, we need to know the address of these

LIST OF TABLES

Appendices

A.4 Multisets

Multisets are basically sets that allow multiple occurrences of each element. Formally a multiset is an ordered pair (A, m) where A is a set, called underlying set, and m : A → N a function giving the multiplicity of each element. Multisets are denoted as ⦃_⦄. For instance, ⦃0, 0, 1⦄ is the multiset

Indeed, two multisets (A, m a ) and (B, m B ) are equal if

elements only in B have multiplicity 0

We can always extend the underlying set by assigning a zero multiplicity to new elements. Sets can be seen as a particular case of multisets where multiplicity is only 0 or 1. We only allow multisets where multiplicity of each element is finite. We do not need more, and infinite multiplicity seems to be much less discussed in the literature. Given a finite set E, we denote

the set of finite multisets whose underlying set is E.

Operations and relations on multisets use the same notations as their set analogs. Let A = (A, m A ), B = (B, m B ) and C = (C, m C ) be multisets. We extend tacitly m A , m B and m C to A ∪ B ∪ C.

We have

In [START_REF] Dershowitz | Proving termination with multiset orderings[END_REF], the union is defined as we defined the sum. We did not make the same choice so that ∪ coincide with the union of sets. Let E a set and a partial order on E. We denote DM , and called Dershowitz-Manna order (or multiset order) induced by , the relation on M (E) defined by

We denote DM the reflexive closure of DM . In other words, to get a smaller multiset, we can remove an occurrence of an element x of B to add as many elements as we want that are lower than x. In [START_REF] Huet | Equations and Rewrite Rules: A Survey[END_REF], there is another, equivalent, definition which is also closer to the interpretation. The alternative definition can be written:

This relation has some good properties. First, DM is a partial order on M (E). Moreover, if is well-founded, then DM is a well-founded order on M (E). The converse is also true, but less interesting for us. These propositions are well-known properties of multiset ordering (see [START_REF] Dershowitz | Proving termination with multiset orderings[END_REF]).

Notation

A.5 Function Theory

We use Bourbaki-style functions. A partial function from A to B is a 3-tuple (A, B, G) where A and B are sets and

The set of total functions from A to B is denoted B A . We can also write f : A → B to define such a function. When f is only a partial function (i.e. a function for which some elements of A may not have an image), we write f : A B. The support of f , denoted supp (f ), is the set of elements of A which have an image, i.e. given a function

The way functions are defined is merely a formalization issue that appears when talking about partial functions and related difficulties. As much as possible, functions will be treated without opening the box.

Given a function f : A B and X a set, we write f |X the restriction of f to X ∩ A, that is, the function from X to B that is defined only on X ∩ supp (f ) and is A.5. FUNCTION THEORY Given a set E, a function f : E → E, and an integer n ∈ N, we write f n the n th iterate of f . Formally

A.7 Local Notations

In this work, we introduce many notations that are not standard. They are recalled here as the user may lose where they come from and what they mean. For each notation, we specify where it was introduced, and we recall briefly its meaning. Notations of each part are mostly independent.

A.7.1 Notations of Part I "Semantics of Mixed C and x86 Assembly"

By-products

T he implementation of the reduced product with ghost variables and assembly han- dling made several by-products as external tools or libraries, that are presented in this appendix. We do not mention libraries that are internal to Astrée.

B.1 OColor

OColor is an OCaml pretty printing library that benefit from semantics tags defined by OCaml's standard module Format. It has a public version available at [Che20b]. The purpose of this library is to emit ANSI escape sequences (see [START_REF] Iso | Control Functions for Coded Character Sets[END_REF]) to produce a nice and more readable output. It is about 4 kloc long and has no run-time dependency.

Among all ANSI escape sequences, we are mainly interested in Select Graphic Rendition (SGR) sequences. These sequences apply styles and are the most frequently used. In the following we will refer to them when writing "ANSI escape sequences". For completeness, we may note that there are other useful ANSI escape sequences, such as those that allow mouse integration, or fine cursor control * . ANSI escape sequences are interpreted completely linearly without memory: each sequence define a change in style that is applied on the current style state. ANSI sequences allow setting the foreground color, the background color, whether the text is underlined, doubly underlined or neither, whether the text is bold. . . . Some styles can be independently reverted, like foreground and background colors, blink, conceal, reverse video, underlined and overlined † . Other styles cannot be reverted, like faint, fraktur, framed, italic and encircled ‡ . When styles cannot be removed independently, we still may reset the current style to the default style, and apply the new formatting we want (that is, the old format, except what we want removed). But unlike L A T E X or HTML, there are no well-parenthesized tags: in L A T E X, writing \textcolor{blue}{a\textcolor{red}{b}c} produces abc. There is no * These sequences are very useful to write text-based user interface (TUI), even if they are often hidden in an adequate library.

† Bold off is specified by ECMA-48 (see [START_REF] Ecma | Control Functions for Coded Character Sets[END_REF]) as doubly underlined, and most shells seem to implement it this way.

‡ And, in practice, bold.

To OCaml". It is named after the Greek philosopher Πλάτων whose work is very long (as Python's standard library) and report the life and philosophy of Socrates, just like Plato brings a very handy library to OCaml's world. Plato is about 5 kloc long. Initially, Plato was started to adapt into OCaml the Python's pathlib library, which is very convenient to work with filenames and files. We were especially interested in the normalization function that removes symbolic links from a path. As it was very helpful, other parts of Python's standard libraries were adapted, in particular datetime, itertools, configparser and their dependencies. Plato also provides functions on list, arrays and strings in place of Python's standard methods. In particular, the slice operator is implemented for all these types, with optimal asymptotic complexity each type allows.

For now, a very small part of Python's standard library is adapted to OCaml. Some parts do not make sense at all, like importlib, gc or inspect. Some libraries have already fair OCaml equivalent, like re or json. Other libraries can still be implemented when needed. An especially nice library to adapt would be argparse as it is very parameterizable and powerful, yet simple to use.

B.4 Socrates

Socrates is by far the largest by-product of Astrée development but also the most specific one. It is a very flexible testing tool that copes with all particularities of Astrée. Socrates stands for "Subtle Output Comparison for Reasonable and Automatic Test Execution for astreeS". It is written in Python and is a bit more than 10 kloc long. The name of this tool was chosen as a reference to the Socratic method: Socrates (Σωκρᾰτης) asked questions until it manages to prove its interlocutor is wrong, often without explaining what is right (and, probably, without knowing it); a testing tool can only exhibit the tested program is wrong by asking it to answer many questions.

Testing Astrée is quite difficult. Since most parts are functors whose parameters have huge signature, unit testing with mocks is not realistic with existing tools. Socrates performs only functional tests: it runs analyses and compares the result with a standard.

Tests are stored in a DAG structure in JSON (JavaScript Object Notation) files. Each file describes nodes, whose children are in the current file or in a subdirectory * * . Nodes in JSON files can also be leaves, which correspond to tests. Each leaf points to at least one C file to analyze. Each node of the DAG stores also configuration, it can be configuration of Astrée (like command-line options) or parameters for Socrates, like the timeout. The configuration with which we run an analysis is collected on the whole path from the root to the leaf. This structure allows each leaf to be accessed from multiple paths. For instance, all pure C test cases are run with assembly handling enabled and disabled. This is easy to achieve using a diamond at the top of the DAG, as shown in Figure B.1. All tests collected from the path on the left have assembly enabled, APPENDIX B. BY-PRODUCTS lead to accepting standards that have been generated for another context, though the command line options are the same. The current solution used in Socrates describes the internal settings we expect in Astrée; standards are identified by these internal settings. Thanks to a configuration file Socrates knows how to generate command line options to reach a given setting. In particular, this configuration file contains default values, so that no flag is generated if only the default value is specified for a parameter. Moreover, the language in this file is quite expressive and allows options to generate internally lists, sets, maps. It also handles command line options that are aliases for several other (with or without argument).

Socrates is also able to handle several versions and branches of Astrée, so that it can be used with both AstréeA and AstréeS on the same test suite. To do so, test cases and configuration options can be annotated with a branch (AstréeA or AstréeS, for instance), and a complex condition on version number. Using these constraints, Socrates can exclude irrelevant tests and generate command-line arguments for different versions of Astrée. In particular, it allows the semantics of a command line option to change between versions. This feature requires Astrée to be able to give its own version and branch. The branch is an arbitrary string that does not contain the character -. Versions number follows the specification of PEP440 ¶ ¶ , which allows very expressive version numbers. Version specifiers are already expressive, but not easy enough to use. Socrates allows arbitrary logic propositions on version numbers, in which atoms are version specifiers as described in PEP440.

Changing the structure of the test suite is a major change, but it should be feasible. For instance, we explained how we started by using a test suite with command-line options, but now we specify internal settings we want. To perform such tasks, Socrates has a migrator framework to easily translate a test suite of an old version to a newer one. Even the migrator is made to be user-friendly as it does its best to perform migrations even indirectly, but with a minimal number of steps.

Socrates' main interface looks like a shell, with entry validation (with explicit error messages), advanced completion (fuzzy and context sensitive), persistent searchable history and common shortcuts. It is important Socrates stays alive between commands as it is stateful. For instance, we can walk on the DAG (using Socrates' cd command) to run a subset of tests. Also, when tests fail, results are in memory, and not written into files until the user decide to do so. It can print nice diffs between the standard and the actual output. . . . In such cases, to examine errors, update standards, and other more interactive tasks, Socrates provides a TUI. Socrates also has a non-interactive mode, but it is more limited as we cannot inspect errors as soon as Socrates is terminated. Its main purpose is to easily check a version is correct, typically using continuous integration.

To complete tests as fast as possible Socrates can detect the number of available cores on the current computer and run tests concurrently. It can also run tests using a test in reasonable time, so we add an option. Later, when debug flags are disabled by default, we would like to remove the command line option otherwise, we simply accumulate a lot of such flags.

¶ ¶ PEP 440 -Version Identification and Dependency Specification, https://www.python.org/dev/p eps/pep-0440/ B.5. VIRGIL 387 version of Astrée it finds on a remote host and which is securely deleted from the local host at the end. It is also able to send a local version of Astrée on a remote host and run tests there, which is interesting if the remote host is more powerful.

Socrates is also able to discover tests in folders and subfolders of its suite. It assists the user to create adequate JSON files, though, manual intervention is still needed to add some configuration. Socrates is aware of symbolic links, handles the gently, and, in particular, is protected against loops * * * . More generally, Socrates knows how to deal with most errors of the external world.

All these features (and many minor ones we skipped) make Socrates a very useful tool, pleasant to use. Since no restart is required when Astrée binary is changed or to select the native or bytecode version, it is common to keep Socrates running for days while developing. For now, it was designed for Astrée only. Hopefully, we believe it can be adapted to test a large variety of software, as long as they use files as input and the correctness of the test can be decided from the output. The main things to change are the parser, the summary comparator and the configuration file of command line options.

B.5 Virgil

Virgil is the more modest by-product of Astrée. Its purpose is to update virtual machine disk images, which are used for continuous integration. Virgil stands for "Virtualbox Image Renewal for Gitlab Integration for the Lazy". It is named after the ancient Roman poet Publius Vergilius Maro, who is also Dante's guide through Hell and Purgatory in the Divine Comedy; indeed, disk images are hellish to manage manually, and we are happy to have a guide through this ordeal. Virgil is written in Python and is about 200 loc long. Even small tools can be most supremely helpful. * * * It should be also protected against loops with hard links, but it is not extensively tested.

MOTS CLÉS

Sémantique, Analyse statique, Analyse de programmes, Interprétation abstraite, Systèmes embarqués critiques, Code C, Assembleur, x86, Assembleur inline, Domaine abstrait, Produit réduit, Variables fantômes RÉSUMÉ Cette thèse est consacrée à l'analyse de logiciels de bas niveau, tels que les systèmes d'exploitation, par interprétation abstraite. L'analyse des OS est une question importante pour garantir la sûreté des systèmes logiciels puisqu'ils forment le niveau immédiatement au dessus du matériel et que toutes les tâches applicatives dépendent d'eux. Pour des applications critiques, on veut s'assurer que l'OS ne plante pas, mais aussi qu'il assure l'isolation des programmes, de sorte qu'un programme dont la fiabilité n'a pas été établie ne puisse perturber un programme de confiance. L'analyse de ce genre de programmes soulève des problèmes spécifiques. Cela provient du fait que les OS doivent contrôler le matériel avec des opérations qui n'ont pas de sens dans un programme ordinaire. De plus, comme les fonctionnalités matérielles sont en dehors du for du C, le code source contient des blocs de code assembleur mêlés au C. Ce sont les deux axes de cette thèse : gérer les mélanges de C et d'assembleur, et abstraire finement les opérations spécifiques aux logiciels de bas niveau. Ce travail est guidé par l'analyse d'un cas d'étude d'un partenaire industriel, ce qui a nécessité l'implémentation des méthodes proposées dans l'analyseur statique Astrée. La première partie s'intéresse à la formalisation d'un langage mélangeant des modèles simplifiés du C et de l'assembleur, depuis la syntaxe jusqu'à la sémantique. Cette spécification est importante pour définir ce qui est légal et ce qui constitue une erreur, tout en tenant compte de la complexité des interactions du C et de l'assembleur, tant en termes de données que de flot de contrôle. La seconde partie est une introduction sommaire à l'interprétation abstraite qui se limite à ce qui est utile par la suite. La troisième partie propose une abstraction de la sémantique des mélanges de C et d'assembleur. Il s'agit en fait d'une collection d'abstractions paramétriques qui gèrent chacun des aspects de cette sémantique. La quatrième partie s'intéresse à l'abstraction des opérations spécifiques aux logiciels de bas niveau. Les propriétés d'intérêt peuvent être facilement prouvées à l'aide de variables fantômes, mais pour des raisons techniques, il est difficile de concevoir un produit réduit de domaines abstraits qui supporte une gestion satisfaisante des variables fantômes. Cette partie construit un tel cadre très général ainsi que des domaines qui permettent de résoudre beaucoup de problèmes dont le nôtre. L'ultime partie présente quelques propriétés à prouver pour garantir l'isolation des programmes, qui n'ont pas été traitées car elles posent de nouvelles et complexes questions. On donne aussi quelques propositions d'amélioration du produit de domaines avec variables fantômes introduit dans la partie précédente, tant en termes de fonctionnalités que de performances.

ABSTRACT

This thesis is dedicated to the analysis of low-level software, like operating systems, by abstract interpretation. Analyzing OSes is a crucial issue to guarantee the safety of software systems since they are the layer immediately above the hardware and that all applicative tasks rely on them. For critical applications, we want to prove that the OS does not crash, and that it ensures the isolation of programs, so that an untrusted program cannot disrupt a trusted one. The analysis of this kind of programs raises specific issues. This is because OSes must control hardware using instructions that are meaningless in ordinary programs. In addition, because hardware features are outside the scope of C, source code includes assembly blocks mixed with C code. These are the two main axes in this thesis: handling mixed C and assembly, and precise abstraction of instructions that are specific to low-level software. This work is motivated by the analysis of a case study emanating from an industrial partner, which required the implementation of proposed methods in the static analyzer Astrée. The first part is about the formalization of a language mixing simplified models of C and assembly, from syntax to semantics. This specification is crucial to define what is legal and what is a bug, while taking into account the intricacy of interactions of C and assembly, in terms of data flow and control flow. The second part is a short introduction to abstract interpretation focusing on what is useful thereafter. The third part proposes an abstraction of the semantics of mixed C and assembly. This is actually a series of parametric abstractions handling each aspect of the semantics. The fourth part is interested in the question of the abstraction of instructions specific to low-level software. Interest properties can easily be proven using ghost variables, but because of technical reasons, it is difficult to design a reduced product of abstract domains that allows a satisfactory handling of ghost variables. This part builds such a general framework with domains that allow us to solve our problem and many others. The final part details properties to prove in order to guarantee isolation of programs that have not been treated since they raise many complicated questions. We also give some suggestions to improve the product of domains with ghost variables introduced in the previous part, in terms of features and performances.

KEYWORDS

Semantics, Static Analysis, Program Analysis, Abstract Interpretation, Critical Embedded Systems, C Code, Assembly, x86, Inline Assembly, Abstract Domains, Reduced Product, Ghost Variables