Dr Sebastian Hack Prüfungsausschuss

Dr Pascal Fontaine

Dr Alberto Griggio

Dr Thomas Sturm

Dr Christoph Weidenbach Vorsitzender

Prof. Dr Sven Apel Akademischer

Dr Benjamin Kiesl

In this thesis, we present new decision procedures for linear arithmetic in the context of SMT solvers and theorem provers: 1) CutSat++, a calculus for linear integer arithmetic that combines techniques from SAT solving and quantifier elimination in order to be sound, terminating, and complete.

2) The largest cube test and the unit cube test, two sound (although incomplete) tests that find integer and mixed solutions in polynomial time. The tests are especially efficient on absolutely unbounded constraint systems, which are difficult to handle for many other decision procedures.

3) Techniques for the investigation of equalities implied by a constraint system. Moreover, we present several applications for these techniques. 4) The Double-Bounded reduction and the Mixed-Echelon-Hermite transformation, two transformations that reduce any constraint system in polynomial time to an equisatisfiable constraint system that is bounded. The transformations are beneficial because they turn branch-and-bound into a complete and efficient decision procedure for unbounded constraint systems.

We have implemented the above decision procedures (except for Cut-Sat++) as part of our linear arithmetic theory solver SPASS-IQ and as part of our CDCL(LA) solver SPASS-SATT. We also present various benchmark evaluations that confirm the practical efficiency of our new decision procedures.

III

IV

First and foremost, I would like to thank my supervisors Christoph Weidenbach and Thomas Sturm, who not only introduced me to automated reasoning and linear arithmetic, but also supported me throughout the different stages of my research. Both gave me the freedom to follow my own research goals and the necessary guidance that enabled me to reach them.

Next, I want to thank my colleagues from the Automation of Logic group for the friendly and welcoming atmosphere, for many interesting and entertaining discussions, as well as for all their good advise. I especially want to thank Mathias Fleury, Fabian Kunze, Michaël Mera, Simon Schwarz, Dominik Wagner, and Christoph Weidenbach for helping me develop SPASS-SATT. I hope they are as proud of our joint work as I am.

Zusammenfassung

In dieser Arbeit präsentieren wir neue Entscheidungsprozeduren für lineare Arithmetik im Kontext von SMT-Solvern und Theorembeweisern: 1) CutSat++, ein korrekter und vollständiger Kalkül für ganzzahlige lineare Arithmetik, der Techniken zur Entscheidung von Aussagenlogik mit Techniken aus der Quantorenelimination vereint. 2) Der Größte-Würfeltest und der Einheitswürfeltest, zwei korrekte (wenn auch unvollständige) Tests, die in polynomieller Zeit (gemischt-)ganzzahlige Lösungen finden. Die Tests sind besonders effizient auf vollständig unbegrenzten Systemen, welche für viele andere Entscheidungsprozeduren schwer sind.

3) Techniken zur Ermittlung von Gleichungen, die von einem linearen Ungleichungssystem impliziert werden. Des Weiteren präsentieren wir mehrere Anwendungsmöglichkeiten für diese Techniken. 4) Die Beidseitig-Begrenzte-Reduktion und die Gemischte-Echelon-Hermitesche-Transformation, die ein Ungleichungssystem in polynomieller Zeit auf ein erfüllbarkeitsäquivalentes System reduzieren, das begrenzt ist. Vereint verwandeln die Transformationen Branch-and-Bound in eine vollständige und effiziente Entscheidungsprozedur für unbeschränkte Ungleichungssysteme.

Wir haben diese Techniken (ausgenommen CutSat++) in SPASS-IQ (unserem theory solver für lineare Arithmetik) und in SPASS-SATT (unserem CDCL(LA) solver) implementiert. Basierend darauf präsentieren wir Benchmark-Evaluationen, die die Effizienz unserer Entscheidungsprozeduren bestätigen.

V VI

List of Figures

1. [START_REF]SMT-COMP 2018 results for QF LIA (main track)[END_REF] The rational solutions of our linear arithmetic example problem 1. [START_REF]SMT-COMP 2018 results for QF LRA (main track)[END_REF] The integer solutions of our linear arithmetic example problem 1. [START_REF]The SPASS Workbench[END_REF] The mixed solutions of our linear arithmetic example problem 1.4 A partially bounded system; the directions h = (-1, 1) T and -h are the only bounded directions in the example . . . . . . 1.5 A bounded system; all directions are bounded . . . . . . . . . 1.6 An absolutely unbounded system; all directions are unbounded 2.1 The pivot and update functions [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]. . . . . . . . . . . . . . .

2.2

The main function of the simplex algorithm [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF] . . . . . . . 2.3 A conflict extraction function for the simplex algorithm [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF] . 2.4 Incremental assert functions for the simplex algorithm [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF] . . 2.5 A simplex backtrack function [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF] . . . . . . . . . . . . . . . . 2.6 A partially bounded system; the directions h = (-1, 1) T and -h are the only bounded directions in the example . . . . . . 2.7 A bounded system; all directions are bounded . . . . . . . . . 2.8 An absolutely unbounded system; all directions are unbounded 2.9 Two series of problems for which branch-and-bound diverges. 5 PivR(k, H) and PivI(k, H) determine the largest column index of a non-zero rational/integer column in H . . . . . . . . 6.6 ExtendR(Hy ≤ u, V, h T m+1 y ≤ b m+1 , j) . . . . . . . . . . . . . 6.7 ExtendI(Hy ≤ u, V, h T m+1 y ≤ b m+1 , j) . . . . . . . . . . . . . 6.8 ReduceLeftI(H , V, p) . . . . . . . . . . . . . . . . . . . . . . 6.9 AbstractToInt(H , V , p) . . . . . . . . . . . . . . . . . . . . [START_REF] Baaz | Normal form transformations[END_REF] Chapter 1

Introduction

Linear arithmetic has been addressed and thoroughly investigated by at least two independent research lines: (i) optimization via linear programming (LP), integer programming (ILP), and mixed real integer programming (MILP) [START_REF] Beale | An alternative method for linear programming[END_REF][START_REF] Bertacco | A feasibility pump heuristic for general mixed-integer problems[END_REF][START_REF] Robert | New finite pivoting rules for the simplex method[END_REF][START_REF] Boyd | Convex Optimization[END_REF][START_REF] Dantzig | Maximization of a Linear Function of Variables Subject to Linear Inequalities[END_REF][START_REF] Ehrgott | Scalarization techniques[END_REF][START_REF] Glover | General purpose heuristics for integer programming -part I[END_REF][START_REF] Glover | General purpose heuristics for integer programming-part II[END_REF][START_REF] Hillier | Efficient heuristic procedures for integer linear programming with an interior[END_REF][START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Khachiyan | Polynomial algorithms in linear programming[END_REF][START_REF] Lemke | The dual method of solving the linear programming problem[END_REF][START_REF] Martin | Preprocessing and probing techniques for mixed integer programming problems[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF], and (ii) first-order quantifier elimination [START_REF] Berman | Precise bounds for Presburger arithmetic and the reals with addition: Preliminary report[END_REF][START_REF] Berman | The complexity of logical theories[END_REF][START_REF] David | Theorem proving in arithmetic without multiplication[END_REF][START_REF] Ferrante | A decision procedure for the first order theory of real addition with order[END_REF][START_REF] Ferrante | The Computational Complexity of Logical Theories[END_REF][START_REF] Michael | Super-exponential complexity of Presburger arithmetic[END_REF][START_REF] Fürer | The complexity of Presburger arithmetic with bounded quantifier alternation depth[END_REF][START_REF] Von | A bound on solutions of linear integer equalities and inequalities[END_REF][START_REF] Grädel | The complexity of subclasses of logical theories[END_REF][START_REF] Lasaruk | Weak quantifier elimination for the full linear theory of the integers. A uniform generalization of Presburger arithmetic[END_REF][START_REF] Oppen | A 2 2 2 pn upper bound on the complexity of Presburger arithmetic[END_REF][START_REF] Presburger | Über die Vollständigkeit eines gewissen Systems der Arithmetik ganzer Zahlen, in welchem die Addition als einzige Operation hervortritt[END_REF][START_REF] Volker | The complexity of almost linear diophantine problems[END_REF]. In this thesis, we are, however, interested in decision procedures for linear arithmetic for a different context: automated reasoning.

To be more precise, we are interested in the satisfiability of linear arithmetic constraint systems, which we simply call problems, in the context of the combination of theories, as they occur, e.g., in SMT (satisfiability modulo theories) solving or theorem proving [START_REF] Althaus | Superposition modulo linear arithmetic SUP(LA)[END_REF][START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF][START_REF] Barrett | Satisfiability Modulo Theories[END_REF][START_REF] Christian Blanchette | Extending Sledgehammer with SMT solvers[END_REF][START_REF] Christian Blanchette | More SPASS with Isabelle -superposition with hard sorts and configurable simplification[END_REF][START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF][START_REF] Christ | Cutting the mix[END_REF][START_REF] Cohen | VCC: A practical system for verifying concurrent C[END_REF][START_REF] Mendonça De Moura | Applications and challenges in satisfiability modulo theories[END_REF][START_REF] Mendonça De Moura | Satisfiability modulo theories: introduction and applications[END_REF][START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF][START_REF] Dragan | Bound propagation for arithmetic reasoning in Vampire[END_REF][START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF][START_REF] Faure | SAT modulo the theory of linear arithmetic: Exact, inexact and commercial solvers[END_REF][START_REF] Ganzinger | DPLL(T): fast decision procedures[END_REF][START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF][START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF][START_REF] Kim | Efficient term-ite conversion for satisfiability modulo theories[END_REF][START_REF] King | Simplex with sum of infeasibilities for SMT[END_REF][START_REF] Nieuwenhuis | The IntSat method for integer linear programming[END_REF][START_REF] Reynolds | Refutation-based synthesis in SMT[END_REF].

From this perspective, the above-mentioned research lines, which are based on optimization and quantifier elimination, address problems that are too general for our purposes: the former considers optimization aspects that go considerably beyond our pure satisfiability problems; the latter considers quantifier alternation, which is more complicated than the purely existentially quantified problems that we consider. It is, therefore, no surprise that the decision procedures for optimization and quantifier elimination do not fit the requirements that make SMT solving and theorem proving efficient. One of those requirements is that SMT solvers and theorem provers solve internally not just one large system of linear inequalities but a large number of incrementally connected, small systems of linear inequalities. Therefore, exploiting the incremental connection is key for making SMT solvers and theorem provers efficient [START_REF] Faure | SAT modulo the theory of linear arithmetic: Exact, inexact and commercial solvers[END_REF]. To be more precise, linear arithmetic decision procedures for SMT solving and automated reasoning have to be incrementally efficient, i.e., gain an advantage from subsequently solving incrementally connected problems, and have to produce conflict explanati-ons for unsatisfiable problems in order to instantly recognize other problems that are unsatisfiable for the same reason. Consequently, the SMT and theorem proving communities have developed several decision procedures that fit these requirements [START_REF] Althaus | Superposition modulo linear arithmetic SUP(LA)[END_REF][START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF][START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF][START_REF] Christ | Cutting the mix[END_REF][START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF][START_REF] Dragan | Bound propagation for arithmetic reasoning in Vampire[END_REF][START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF][START_REF] Faure | SAT modulo the theory of linear arithmetic: Exact, inexact and commercial solvers[END_REF][START_REF] Ganzinger | DPLL(T): fast decision procedures[END_REF][START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF][START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF][START_REF] Kim | Efficient term-ite conversion for satisfiability modulo theories[END_REF][START_REF] King | Simplex with sum of infeasibilities for SMT[END_REF][START_REF] Nieuwenhuis | The IntSat method for integer linear programming[END_REF].

In this thesis, we present our own linear arithmetic decision procedures for SMT solving and theorem proving. The focus of our decision procedures are (i) problem classes that appear in SMT solving and theorem proving but are ignored by other research lines, (ii) problem classes that were previously considered too hard to solve efficiently in practice, and (iii) problem classes on which most existing implementations diverge. Moreover, we tried to design our decision procedures (a) to produce conflict (and other) explanations, (b) to be incrementally efficient, (c) to be practically efficient, and (d) to be easy to combine with other techniques.

The remainder of the introduction is organized as follows: First, we quickly introduce the theory of linear arithmetic and its subtheories in Section 1.1. Then, we explain in Section 1.2 how linear arithmetic fits into the context of automated reasoning (e.g. via SMT solving and theorem proving). In Section 1.3, we explain unbounded problems, which is the class of linear arithmetic problems that most of our decision procedures target and that is difficult to handle for most existing decision procedures. Then, we explain in Section 1.4 how we measure the efficiency of our decision procedures. The last section, Section 1.5, gives an overview over our own contributions to linear arithmetic decision procedures, which we then present in more detail in the remainder of this thesis.

The Theory of Linear Arithmetic

A first-order theory defines a first-order language, i.e., a set of first-order formulas, and their interpretation. The theory of linear arithmetic is such a first-order theory and it defines the set of linear arithmetic formulas, i.e., the set of formulas consisting of all Boolean combinations of linear inequalities over arithmetic variables [START_REF] Ferrante | A decision procedure for the first order theory of real addition with order[END_REF][START_REF] Presburger | Über die Vollständigkeit eines gewissen Systems der Arithmetik ganzer Zahlen, in welchem die Addition als einzige Operation hervortritt[END_REF][START_REF] Sebastiani | Lazy satisability modulo theories[END_REF]. This means the most basic linear arithmetic formulas are linear inequalities a i1 • x 1 + . . . + a in • x n ≤ b n , where a i1 , . . . , a in , b i ∈ Q are rational constants and x 1 , . . . , x n are typed arithmetic variables. All other linear arithmetic formulas are constructed inductively through Boolean combinations, i.e., ¬φ, φ ∧ ψ, φ ∨ ψ, φ → ψ, φ ↔ ψ, ∀x i .φ, and ∃x i .φ are linear arithmetic formulas if φ and ψ are linear arithmetic formulas and if x i and x j are arithmetic variables typed as an integer variable and a rational variable, respectively. The theory of linear arithmetic also defines the interpretation of linear arithmetic formulas: all arithmetic functions, predicates, and constants (i.e., +, •, ≤ and a ij ∈ Q) are interpreted by their well-known mathematical definitions and all Boolean connectives are interpreted as we are used to from first-order logic.

The majority of decision procedures that we present in this thesis handle on their own only the ground conjunctive subset of linear arithmetic, i.e., the subset where all Boolean combinations are conjunctions and all variables are free but treated as if they were existentially quantified. For simplicity, we call the formulas belonging to this subset problems. Nevertheless, our decision procedures can solve much more expressive subsets of linear arithmetic if we use them in a modular way inside an SMT solver, which is the focus of Chapter 8. Using our decision procedures in a typical SMT solver framework (i.e., CDCL(T) [START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF][START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF][START_REF] Ganzinger | DPLL(T): fast decision procedures[END_REF][START_REF] Nieuwenhuis | DPLL(T) with exhaustive theory propagation and its application to difference logic[END_REF][START_REF] Nieuwenhuis | Solving SAT and SAT modulo theories: From an abstract Davis-Putnam-Logemann-Loveland procedure to DPLL(T )[END_REF]) generates a decision procedure for linear arithmetic formulas that are ground and in CNF (clause normal form). For simplicity, we call such formulas CNF problems. 1Let us now look at an example of a problem:

-5 • x 1 + 2 • x 2 ≤ 0 ∧ 4 • x 1 -3 • x 2 ≤ 0 ∧ 5 • x 1 + 2 • x 2 ≤ 15 ∧ -3 • x 1 -2 • x 2 ≤ -4
This problem contains four different inequalities and two different variables. Since our problems contain only one type of Boolean connective, we also often omit the conjunction symbols ∧ and denote them instead by a set of inequalities (also called a system of inequalities):

{ -5 • x 1 + 2 • x 2 ≤ 0, 4 • x 1 -3 • x 2 ≤ 0, 5 • x 1 + 2 • x 2 ≤ 15, -3 • x 1 -2 • x 2 ≤ -4
} Now given such a problem, our goal is to determine whether there exists an assignment for all of our variables that satisfies each of the inequalities.

Such an assignment is also called a solution to our problem. However, the solutions which we accept and the complexity of finding them is also dependent on the types of our variables. For instance, if both of our variables x 1 and x 2 are rational variables, then all points contained in the polyhedron shown in Figure 1.1 are solutions; if both of our variables x 1 and x 2 are integer variables, then only the point (x 1 , x 2 ) = (1, 2) highlighted by the circle in Figure 1.2 is a solution; and if x 1 is a rational variable and x 2 an integer variable, then all points on the lines shown in Figure 1.3 are solutions.

The different acceptable solutions are also the reason, why we generally partition our problems into three subtheories of linear arithmetic: if our problem contains only rational variables, then it belongs to the theory of linear rational arithmetic (LRA), if our problem contains only integer variables, then it belongs to the theory of linear integer arithmetic (LIA), and if our problem contains both types of variables, then it belongs to the theory of linear mixed arithmetic (LIRA). Finding a solution for a LIA or LIRA problem is an NP-complete task [START_REF] Karp | Reducibility among combinatorial problems[END_REF][START_REF] Papadimitriou | On the complexity of integer programming[END_REF]. Finding a solution for an LRA problem is a much easier task and can be accomplished in polynomial time [START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Khachiyan | Polynomial algorithms in linear programming[END_REF]. This means that our three subtheories have different levels of complexity (at least if we assume that P = NP).

Linear Arithmetic in Automated Reasoning

Automated reasoning is a research area that is dedicated to teaching computers how to reason autonomously. This is accomplished by encoding reasoning problems as logical formulas and by (dis-)proving the resulting formulas via automated reasoning programs, e.g., SMT solvers and theorem provers. Since most problems can be encoded as logical formulas (although not necessarily as first-order formulas), the potential applications for automated reasoning are infinite. The applications are, however, limited by the undecidability of some logics and the efficiency of the existing automated reasoning programs. Practically relevant example applications for automated reasoning appear in analysis, testing, verification, and synthesis of hard-and software, as well as in interactive theorem proving [START_REF] Barrett | Satisfiability Modulo Theories[END_REF][START_REF] Christian Blanchette | Extending Sledgehammer with SMT solvers[END_REF][START_REF] Christian Blanchette | More SPASS with Isabelle -superposition with hard sorts and configurable simplification[END_REF][START_REF] Cohen | VCC: A practical system for verifying concurrent C[END_REF][START_REF] Mendonça De Moura | Applications and challenges in satisfiability modulo theories[END_REF][START_REF] Mendonça De Moura | Satisfiability modulo theories: introduction and applications[END_REF][START_REF] Reynolds | Refutation-based synthesis in SMT[END_REF].

Linear arithmetic is relevant to automated reasoning because many target applications for automated reasoning use arithmetic on their own. For instance, most programs use arithmetic variables (e.g., integers) and perform arithmetic operations on those variables. Therefore, software verification has to deal with arithmetic; often in combination with other theories, e.g., for arrays and other data structures [START_REF] Mccarthy | Towards a mathematical science of computation[END_REF]. This is also the reason why there exist many approaches that combine automated reasoning programs for different theories in a modular way. Examples of such modular approaches are CDCL(T) [START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF][START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF][START_REF] Ganzinger | DPLL(T): fast decision procedures[END_REF][START_REF] Nieuwenhuis | DPLL(T) with exhaustive theory propagation and its application to difference logic[END_REF][START_REF] Nieuwenhuis | Solving SAT and SAT modulo theories: From an abstract Davis-Putnam-Logemann-Loveland procedure to DPLL(T )[END_REF] (conflict-driven clause-learning modulo theo-ries) for SMT solving and SUP(T) (superposition modulo theories) [START_REF] Althaus | Superposition modulo linear arithmetic SUP(LA)[END_REF][START_REF] Baumgartner | Hierarchic superposition with weak abstraction[END_REF][START_REF] Fietzke | Superposition as a decision procedure for timed automata[END_REF] for theorem proving. CDCL(T), for instance, abstracts the input formula into a propostional formula and repeatedly selects propositionally satisfiable models with a SAT solver, until a (combined) theory solver verifies that one of the abstracted propositional models is also a theory satisfiable model.

Modular approaches can also be used to extend less expressive theory solvers. For instance, a theory solver for CDCL(LA) only has to handle problems and the CDCL(LA) framework extends it to a solver for all CNF problems. Moreover, a combined theory solver, i.e., a theory solver handling multiple theories, can be created with the Nelson-Oppen method [START_REF] Nelson | Simplification by cooperating decision procedures[END_REF].

Theory solvers also have to fulfill certain requirements or the modular approach is not efficient enough in practice [START_REF] Faure | SAT modulo the theory of linear arithmetic: Exact, inexact and commercial solvers[END_REF]. To be more precise, theory solvers for SMT solving and theorem proving have to be incrementally efficient, i.e., gain an advantage from subsequently solving incrementally connected problems, and have to produce conflict explanations(certificates of unsatisfiability) for unsatisfiable problems in order to instantly recognize other problems that are unsatisfiable for the same reason. Also important are certificates of satisfiability (e.g., solution assignments) and theory propagation and theory learning capabilities.

We are interest in decision procedures for linear arithmetic in the context of automated reasoning because of a possible combination with superposition via the SUP(T) approach [START_REF] Althaus | Superposition modulo linear arithmetic SUP(LA)[END_REF][START_REF] Baumgartner | Hierarchic superposition with weak abstraction[END_REF][START_REF] Fietzke | Superposition as a decision procedure for timed automata[END_REF]. In the superposition context, the modular theory solver often has to handle so-called unbounded problems, which are particularly hard to solve linear arithmetic problems. Unbounded problems occur so often in SUP(T) because (i) SUP(T) isolates linear arithmetic inequalities to their respective clauses, i.e., inequalities belonging to different clauses interact only under very specific conditions; and (ii) the inequalities that bound an existential variable are often split over multiple clauses (especially in the ground case of SUP(T)) [START_REF] Althaus | Superposition modulo linear arithmetic SUP(LA)[END_REF].

Unbounded problems also appear in other areas of automated reasoning, although less frequently. Either because of bad encodings, necessary but complicating transformations, e.g., slacking (see Chapter 6.5), or the sheer complexity of the verification goal. Hence, efficient techniques for handling unbounded problems are necessary for a generally reliable combined procedure and their development became the focus of this thesis.

Unbounded Problems

A direction h ∈ Q n is bounded in a problem if there exist two constants l, u ∈ Q such that all rational solutions s ∈ Q n of the problem fulfill the constraints l ≤ h T s ≤ u. A direction h ∈ Q n is unbounded in a problem if no such two constants exist. (See Figures 1. 4-1.6 for examples of (un)bounded directions.) Based on these definitions, a problem is bounded if all directions are bounded and a problem is unbounded if at least one direction is unbounded.

Unbounded directions become interesting when we consider termination of decision procedures for linear integer (and mixed) arithmetic. Achieving termination on a bounded problem is relatively easy because such a problem imposes bounds on (all of) its variables x i . Therefore, there only exist finitely many potential integer solutions, which can be enumerated to find an actual integer solution. The efficiency of the decision procedure then mostly depends on the size of the variable bounds and on the order of our enumeration. Achieving termination on unbounded problems is harder because the search space is not finite. Most decision procedures, e.g., branch-andbound [START_REF] Ailsa | An automatic method for solving discrete programming problems[END_REF][START_REF] John | An algorithm for the traveling salesman problem[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF], will even diverge without additional help (see also Example 2.8.3 in Chapter 2). However, the additional help described in the literature is often not valuable because it does not scale in practice. As an example, let us look at the a priori bounds presented by Papademitriu [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]: 2 A priori bounds intersect the original problem with a cube that is so large that the problem only has an integer (or a mixed) solution outside of the cube if it also has one inside the cube. Therefore, a priori bounds reduce the search space for enumeration based decision procedures, e.g., branch-and-bound, to a finite search space. Hence, enumeration based decision procedures are guaranteed to terminate. However, a priori bounds often describe a search space that is so large that it cannot be explored in reasonable time. For instance, the a priori bounds for the example depicted in Figure 1.4 are in the order of billions. So a priori bounds are valuable in theory but not in practice.

Other termination tactics from the literature, e.g., exhaustive cutting planes [START_REF] Gomory | Outline of an algorithm for integer solutions to linear programs and an algorithm for the mixed integer problem[END_REF] and quantifier elimination [START_REF] David | Theorem proving in arithmetic without multiplication[END_REF], do not scale in practice because of similar arguments. It is, therefore, no surprise that none of these tactics have been implemented in any of the state-of-the-art SMT solvers [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].

Measuring Efficiency

In this thesis, we estimate the practical efficiency of our decision procedures by implementing them inside a theory solver or CDCL(LA) solver and testing them on a large set of benchmarks. To be more precise, a solver is more efficient in practice than another solver (i) if it solves more problems from a relevant benchmark set within a reasonable time limit than the other solver; or (ii) if it solves problems from a relevant benchmark set faster than the other solver. Similarly, a decision procedure helps in practice if its addition to a solver makes the solver more efficient in practice. For our purposes, the relevant benchmark sets are subsets of the SMT-LIB (Satisfiability Modulo Theories Library) benchmarks [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF] and our reasonable time limit is 40 minutes 3 . Note, however, that there exists no perfect method for estimating practical efficiency. Benchmark evaluations are imperfect because we can only test a finite set of benchmarks although there is an infinite number of problems. Therefore, benchmark evaluations are almost guaranteed to be biased. In some sense, this bias is even expected and necessary because our decision procedures are handling NP-hard problem classes and therefore cannot scale on all problems (unless P = NP). As a compromise, we are focusing our benchmarks and decision procedures on (types of) problems that are relevant for applications.

Contributions

The contributions that we present in this thesis are new linear arithmetic decision procedures for SMT solving and theorem proving. We developed these decision procedures in order to efficiently handle unbounded problems. And yet, some of our decision procedures have additional applications beyond unbounded problems. For instance, one of our techniques can be used to investigate and eliminate equalities, to eliminate quantifiers, and to compute all pairs of equivalent variables inside a constraint system, which are necessary for the Nelson-Oppen style combination of theories [START_REF] Nelson | Simplification by cooperating decision procedures[END_REF].

The presentation of our contributions is organized as follows: First, we present in Chapter 2 some important definitions and theorems from the literature and our preferred notations and representations. Next, we present in Chapters 3-6 our new decision procedures for linear arithmetic. Then, we outline in Chapter 7 the implementation of our linear arithmetic theory solver SPASS-IQ that efficiently combines our new decision procedures with other decision procedures for linear arithmetic (e.g., with branch-and-bound and a version of the dual simplex algorithm). In Chapter 8, we then extend SPASS-IQ to a CDCL(LA) solver, which we call SPASS-SATT. Both chapters also highlight all factors that have a major impact on the efficiency of our implementations. These factors include the impact of (i) the chosen decision procedures; (ii) the optimizations that we made to the chosen decision procedures (e.g., optimized data structures and heuristics); (iii) the different interaction techniques between theory solver and SAT solver in our CDCL(LA) framework; and (iv) specialized preprocessing techniques (e.g., for if-then-else expressions and pseudo-boolean inequalities). We conclude the thesis in Chapter 9 with a summary of the presented results. Works related to the contributions of this thesis are summarized in the first section of the relevant chapters. The specific contributions that we present in this thesis are listed below.

Chapter 3: The CutSat++ Calculus

CutSat++, which we present in Chapter 3, is a calculus for linear integer arithmetic that combines techniques from SAT solving and quantifier elimination. It is an extension and refinement of the CutSat calculus by Jovanović and de Moura [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. Compared to its predecessor, CutSat++ terminates on all problems for linear integer arithmetic even without relying on a priori bounds [START_REF] Papadimitriou | On the complexity of integer programming[END_REF] for termination. With a CDCL style calculus CutSat++ efficiently handles problems over guarded variables, i.e., variables with a constant upper and lower bound. On problems with unguarded variables (e.g. unbounded problems) the CDCL style calculus alone is not guaranteed to terminate. Hence, we combine it with a lazy quantifier elimination procedure that transforms a problem containing unguarded variables into one where feasibility depends only on guarded ones. The quantifier elimination procedure is called lazy because we only apply it to conflicts encountered by the CDCL style algorithm. This allows us to avoid certain cases of worst-case exponential behavior that we would otherwise observe by using a quantifier elimination procedure alone. Overall, CutSat++ is a sound, terminating, and complete decision procedure for linear integer arithmetic. Moreover, CutSat++ leaves enough space for model assumptions and simplification rules in order to be efficient in practice.

Chapter 3 is based on a publication with Thomas Sturm and Christoph Weidenbach as co-authors [START_REF] Bromberger | Linear integer arithmetic revisited[END_REF]. An extended version of this publication titled "A Complete and Terminating Approach to Linear Integer Solving" has been accepted for publication in the Journal of Symbolic Computation.

Chapter 4: The Largest Cube Test and the Unit Cube Test

The largest cube test and the unit cube test, which we present in Chapter 4, are two sound (although incomplete) tests that find integer and mixed solutions for linear arithmetic constraint systems in polynomial time. In contrast to many complete methods that search along the problem surface for a solution, these tests use axis-parallel hypercubes, which we simply call cubes, to explore the interior of the problem. To this end, the largest cube test finds a cube with maximum edge length contained in the rational solutions of the input problem, determines its rational valued center, and rounds it to a potential mixed/integer solution. The unit cube test determines instead whether the the rational solutions of the input problem contain a cube with edge length one, which is the minimal edge length that always guarantees that a cube contains a mixed/integer solution.

The tests are especially efficient on constraint systems with a large number of integer solutions, e.g., those that are absolutely unbounded. Inside the SMT-LIB (Satisfiability Modulo Theories Library) benchmarks [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF], we have found almost one thousand problem instances that are absolutely unbounded. Experimental results confirm that our tests are superior on these instances compared to several state-of-the-art SMT solvers.

Chapter 4 is based on two publications with Christoph Weidenbach as co-author [START_REF] Bromberger | Fast cube tests for LIA constraint solving[END_REF][START_REF] Bromberger | New techniques for linear arithmetic: cubes and equalities[END_REF].

Chapter 5: Techniques for the Investigation of Equalities

We present in Chapter 5 new techniques for the investigation of equalities implied by a system of linear arithmetic constraints. The main technique presented in this chapter computes a basis for all implied equalities, i.e., a finite representation of all equalities implied by the linear arithmetic constraints. The equality basis has several applications. We can use it (i) to determine whether a constraint system implies any equality, (ii) to verify whether a constraint system implies a given equality, (iii) to eliminate the equalities from our constraint system, (iv) to eliminate quantifiers, (v) to determine the (un)bounded directions in our constraint system, and (vi) to compute all pairs of equivalent variables inside a constraint system, which are necessary for the Nelson-Oppen style combination of theories [START_REF] Nelson | Simplification by cooperating decision procedures[END_REF].

Chapter 5 is based on two publications with Christoph Weidenbach as co-author [START_REF] Bromberger | Computing a complete basis for equalities implied by a system of LRA constraints[END_REF][START_REF] Bromberger | New techniques for linear arithmetic: cubes and equalities[END_REF].

Chapter 6: A New Bounding Transformation

As mentioned before, many existing approaches for linear mixed and integer arithmetic, e.g., branch-and-bound [START_REF] Ailsa | An automatic method for solving discrete programming problems[END_REF][START_REF] John | An algorithm for the traveling salesman problem[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF], terminate only on bounded problems. Therefore, we present in Chapter 6 a new bounding transformation, i.e., a transformation that reduces any problem to an equisatisfiable problem that is bounded. Our bounding transformation consists of two other transformations: the Mixed-Echelon-Hermite transformation by Christ and Hoenicke [START_REF] Christ | Cutting the mix[END_REF] and our own Double-Bounded reduction. Compared to previous bounding transformations from the literature (e.g., a priori bounds [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]), ours is not only valuable in theory, i.e., the transformation takes only polynomial time, but also in practice, i.e., the resulting problems are solvable in reasonable time. Our transformation is so efficient because it orients itself on the structure of the input problem instead of computing a priori (over-)approximations out of the available constants. Experiments provide further evidence to the efficiency of the transformations in practice. Moreover, we present a polynomial method for converting certificates of (un)satisfiability from the transformed to the original problem.

Chapter 6 is based on [START_REF] Bromberger | A reduction from unbounded linear mixed arithmetic problems into bounded problems[END_REF]. The paper has no coauthors.

Chapter 7: The Implementation of SPASS-IQ SPASS-IQ is the linear arithmetic theory solver that we have implemented as part of our SPASS Workbench [START_REF]The SPASS Workbench[END_REF]. It efficiently combines most of our new decision procedures with existing techniques for linear arithmetic (viz., dual simplex, branch-and-bound, simple rounding, and bound refinement). In Chapter 7, we outline and evaluate SPASS-IQ's overall implementation. Since we already present our new decision procedures as stand-alone procedures in other chapters, this chapter will focus on their combination and any additional implementation factors that have a major impact on the efficiency of SPASS-IQ. The impact of these factors is measured through various benchmark evaluations on the SMT-LIB benchmarks for QF LIA (quantifier free linear integer arithmetic) and QF LRA (quantifier free linear rational arithmetic) [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF]. Parts of Chapter 7 are summarized in [START_REF] Bromberger | SPASS-SATT a CDCL(LA) solver[END_REF]. Coauthors are Mathias Fleury, Simon Schwarz, and Christoph Weidenbach.

Chapter 8: The Implementation of SPASS-SATT SPASS-SATT is an automated reasoner for ground linear arithmetic (with arbitrary boolean combinations) that we have implemented as part of our SPASS Workbench [START_REF]The SPASS Workbench[END_REF]. To be more precise, we combine in SPASS-SATT a CDCL(T) implementation with our theory solver SPASS-IQ to get a CDCL(LA) implementation. The CDCL(LA) implementation still handles only CNF problems (i.e., ground linear arithmetic formulae in clause normal form). We resolve this by adding some preprocessing techniques that turn any ground linear arithmetic formula into an equisatisfiable CNF problem.

In Chapter 8, we explain the construction of SPASS-SATT in more detail. First, we outline the interaction between the theory solver and the SAT solver in the CDCL(LA) implementation. This also includes extensions to the theory reasoning that enhance and guide the search of the SAT solver. Then, we explain the preprocessing techniques that we incorporated into SPASS-SATT. As part of our explanations, we also measure the impact of our theory reasoning extensions and preprocessing techniques through various benchmark evaluations on the SMT-LIB benchmarks for QF LIA (quantifier free linear integer arithmetic) and QF LRA (quantifier free linear rational arithmetic) [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF].

As a consequence of our excellent benchmark results, our decision procedures have been reimplemented in several state-of-the-art SMT solvers. For instance, the unit cube test has been reimplemented in MathSAT5 [START_REF] Cimatti | The MathSAT5 SMT solver[END_REF], SMT-RAT [START_REF] Corzilius | SMT-RAT: an open source C++ toolbox for strategic and parallel SMT solving[END_REF], and Z3 [START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF]. Moreover, we participated with SPASS-SATT in the main track of the 13th International Satisfiability Modulo Theories Competition (SMT-COMP 2018) and ranked first in the category QF LIA (quantifier free linear integer arithmetic) [START_REF]SMT-COMP 2018 results for QF LIA (main track)[END_REF] and ranked second in the category QF LRA (quantifier free linear rational arithmetic) [START_REF]SMT-COMP 2018 results for QF LRA (main track)[END_REF].

Parts of Chapter 8 are summarized in [START_REF] Bromberger | SPASS-SATT a CDCL(LA) solver[END_REF]. Coauthors are Mathias Fleury, Simon Schwarz, and Christoph Weidenbach.

11

Chapter 2

Preliminaries

In this chapter, we present the foundations for this thesis. This includes some well-known theorems and definitions from the literature of linear algebra, linear arithmetic, satisfiability modulo theories, and transition systems. It also includes the notations and representations most commonly used during this thesis.

The chapter is outlined as follows: We start in Section 2.1 with our notations for basic linear algebraic constructs, e.g., vectors and matrices. We also define some general norm and distance functions in the same section. In Section 2.2, we present the syntax and semantics of general arithmetic constraint systems. This includes: several representations for arithmetic constraints; definitions for assignments, solutions, satisfiability, equivalence, and equisatisfiability of constraint systems; as well as definitions for substitutions over constraint systems.

In Section 2.3, we explain how to reduce all of our general arithmetic constraints to non-strict inequalities. We do so because the linear arithmetic literature typically supports only non-strict inequalities [START_REF] Boyd | Convex Optimization[END_REF][START_REF] Hillier | Duality Theory and Sensitivity Analysis[END_REF][START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Lemke | The dual method of solving the linear programming problem[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF]. This also allows us to define our standard (theoretical) input problems as systems of inequalities in Section 2.4. In the same section, we also define an alternative input format, the tableau representation, which we use for algorithms in the context of SMT solver implementations.

In Section 2.5, we define implied constraints and explain their relationship to combinations of linear inequalities. This relationship is also the basis for most formal proofs in linear arithmetic as well as this thesis. Moreover, they allow us to define some interesting properties of constraint systems, e.g., (un)bounded directions, which we explain in Section 2.8.

All techniques presented in this thesis are meant to be used inside SMT solvers. In Section 2.6, we explain CDCL(T), the framework most SMT solvers are based on. Moreover, we formulate most of our contributions as natural extensions of the linear arithmetic decision procedures implemented in most SMT solvers: a version of the dual simplex algorithm for linear rational arithmetic and a branch-and-bound method for linear integer/mixed arithmetic. We present these standard procedures and some popular extensions to them in Section 2.7.

The standard arithmetic decision procedures are efficient on many problems. There does, however, exist a class of problems, where standard decision procedures for linear arithmetic do not guarantee termination or take too much time in practice. We call this class of problems unbounded problems and formally define it in Section 2.8. Since one of the goals of this thesis is to find complete but efficient decisions procedures for linear arithmetic, many of our contributions focus on unbounded problems.

Another way of extracting information from a system of inequalities is to view it as a geometric object. In Chapter 4, we show how to use simpler geometric objects to find integer/mixed solutions for some of our systems. The objects we use for this purpose are defined in Section 2.9.

We describe the majority of our algorithms as pseudocode written in an abstract while-language. We do so because (i) most of our contributions are designed as extensions of existing procedures that were presented in an abstract while-language; and (ii) a pseudocode description is already relatively close to an actual implementation of the respective algorithm. There is, however, one algorithm which we prefer to describe as a transition system instead of pseudocode. This algorithm is CutSat++, our terminating extension to the CutSat algorithm. We choose to present CutSat++ as a transition system because (i) the CutSat calculus was also originally presented as a transition system [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF][START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], and (ii) it is a good representation for proving termination, soundness, and completeness of a complex algorithm. We present the basics of transition systems in Section 2.10.

Basics of Linear Algebra

The linear algebraic notation in this thesis follows [START_REF] Korte | Combinatorial Optimization: Theory and Algorithms[END_REF] with some minor exceptions. 1 While the difference between matrices, vectors, and their components is always clear in context, we generally use upper case letters for matrices (e.g., A), lower case letters for vectors (e.g., x), and lower case letters with an index i or j (e.g., b i , x j ) as components of the associated vector at position i or j, respectively. The only exceptions are the row vectors a T i = (a i1 , . . . , a in ) of a matrix A = (a 1 , . . . , a m ) T , which already contain an index i that indicates the row's position inside A. We also abbreviate the n-dimensional origin (0, . . . , 0) T as 0 n and the n-dimensional all-ones vector (1, . . . , 1) T as 1 n . Moreover, we denote by piv(A, j) the row index of the pivot of a column j, i.e., the smallest row index i with a non-zero entry a ij or m + j if there are no non-zero entries in column j.

Norms

Some of our techniques require the use of d-norms . d [START_REF] Ehrgott | Scalarization techniques[END_REF]. These d-norms are defined as functions ( . d : R n → R) for d ≥ 1 such that

x d = |x 1 | d + . . . + |x n | d 1/d
. There are two d-norms that are especially relevant to this thesis: the 1-norm and the maximum norm. The 1-norm simplifies our general d-norm definition to x 1 = (|x 1 | + . . . + |x n |). The maximum norm is a special d-norm because it is defined by the limit of . d for d → ∞:

x ∞ = max {|x 1 |, . . . , |x n |}. We chose to focus on the 1-norm and the maximum norm because any rational vector b ∈ Q n has a rational 1-norm b 1 ∈ Q and a rational maximum norm b ∞ ∈ Q. This becomes relevant when we later use it to define our fast cube tests (Chapter 4).

Distance

We can also use d-norms to define distance functions dist d (x, y) between two points x, y ∈ R n : dist d (x, y) = xy d [START_REF] Ehrgott | Scalarization techniques[END_REF]. In this thesis, we are rarely interested in an actual distance between points. We use, however, distances to define other relationships between points. For instance, we define a closest integer for a point x as a point x ∈ Z n with minimal distance dist d (x, x ). We also define the operators x j and x such that they describe a closest integer for x j and x, respectively [START_REF] Fischetti | The feasibility pump[END_REF][START_REF] Keith | An atlas of functions: with equator, the atlas function calculator[END_REF]. Formally, this means that x = ( x 1 , . . . , x n ) T and

x j = x j if x jx j < 0.5 , x j if x jx j ≥ 0.5 . This definition of x is also known as simple rounding.

Lemma 2.1.1 (Closest Integer). x is a closest integer to x, i.e.:

∀x ∈ Z n . dist d (x, x ) ≤ dist d (x, x ) .

Proof. We first look at the one-dimensional case, where x j d simplifies to |x j |: ∀x j ∈ Z. |x jx j | ≤ |x jx j | . For x j , x j ∈ Z, there exists z j ∈ Z such that x j = x jz j . Moreover, there exists a d j ∈ [-0.5, 0.5] for each x j such that d j := x jx j . The inequality trivially holds for z j = 0: |x jx j | = |x jx j + z j | = |x jx j | , Next, we use the triangle inequality to get the following relationship for the remaining z j = 0 :

|x j -x j | = |x j -x j + z j | = |d j + z j | ≥ |z j | -|d j | .
Since z j = 0 and d j ∈ [-0.5, 0.5] imply |z j | ≥ 1 and |d j | ≤ 0.5, respectively, we get:

|x j -x j | ≥ |z j | -|d j | ≥ 1 -|d j | ≥ 0.5 ≥ |d j | = |x j -x j | .
The multidimensional case follows from the d-norms' componentwise monotonicity [START_REF] Ehrgott | Scalarization techniques[END_REF], i.e., if |x jx j | ≤ |x jx j | for all j ∈ {1, . . . , n}, then xx d ≤ xx d .

A generalization of the closest integer is the closest k-mixed point to the point x, i.e., the point x with minimal distance dist d (x, x ) such that the last k components of x are integer values. As with the closest integer, we can find a closest k-mixed point with the help of rounding, but we only round the last k components. We define this through the mixed simple rounding function x k := (x 1 , . . . , x n-k , x n-k+1 , . . . , x n ) T . Lemma 2.1.2 (Closest k-Mixed Point). x k is a closest k-mixed point to x.

Proof. Works analogously to the proof of Lemma 2.1.1.

Basics of Linear Arithmetic

The input problems for linear arithmetic are systems of constraints C, i.e., finite sets of constraints corresponding to and sometimes used as conjunctions over their elements. The standard constraints for linear arithmetic are non-strict inequalities a i1 x 1 + . . . + a in x n ≤ b i and strict inequalities a i1 x 1 + . . . + a in x n < b i . In some cases, we also have to handle a third type of constraint, a so-called divisibility constraint d i | a i1 x 1 + . . . + a in x nb i . 2Note, however, that divisibility constraints are introduced only during the execution of some of our algorithms (CutSat, CutSat++, and CutSat g ). Our actual input problems never contains divisibility constraints. Therefore, we only have to handle them explicitly in algorithms where they are introduced, which happens only in the algorithms presented in Chapter 3.

In all of the above constraints, the a ij are constant rational values called coefficients, the b i are constant rational values called constraint bounds, the d i are constant positive integer values called constraint divisors, and the x j are distinct variables, i.e., the variables x j and x i are different unless i = j. Each of those variables is also assigned a type: either rational or integer . If our problem contains only integer variables, then it belongs to the theory of linear integer arithmetic (LIA). If our problem contains only rational variables, then it belongs to the theory of linear rational arithmetic (LRA). If our problem contains both types of variables, then it belongs to the theory of linear mixed arithmetic (LIRA).

For easier access to the components of our constraints and problems, we also define the following functions: coeff(I, x j ) := a ij denotes the coefficient of variable x j in the constraint I, where I := a i1 x 1 + . . . + a in x n ≤ b i or I := a i1 x 1 + . . . + a in x n < b i or I := d i | a i1 x 1 + . . . + a in x nb i ; vars(C) := {x 1 , . . . , x n } denotes the set of all variables that appear in the problem C, i.e., that appear with a non-zero coefficient in any of its constraints; similarly, Qvars(C) and Zvars(C) denote the sets of rational and integer variables that appear in the problem C, respectively. Due to convenience, we assume that the first

Constraint Representations

We previously defined constraints as either non-strict inequalities of the form a i1 x 1 + . . . + a in x n ≤ b i , strict inequalities of the form a i1 x 1 + . . . + a in x n < b i , and divisibility constraints of the form d i | a i1 x 1 + . . . + a in x nb i . We say that a constraint is in standard representation if it is in one of those three forms. However, not all linear constraints fit into this standard representation, e.g., 3 ≥ 2(x 3 +2(x 1 -x 2 )-5). Still, all of them can be transformed into it by using basic mathematical properties, e.g., associativity, commutativity and distributivity. For instance, 3 ≥ 2(x 3 + 2(x 1x 2 ) -5) can be transformed into 4x 1 -4x 2 + 2x 3 ≤ 13. Whenever we construct a new constraint, we assume that it is transformed into an equivalent constraint in the standard representation.

Vector Representation and Focused Representation

The standard representation is a sometimes unnecessarily long representation for abstract constraints. For this reason, we additionally use two alternative representations: the vector representation and the focused representation. Both alternative representations are essentially the standard representation except for some form of abbreviation.

In the vector representation, we represent our coefficients and variables as vectors a i = (a i1 , . . . , a in ) T and x = (x 1 , . . . , x n ) T , respectively. This allows us to abbreviate their sum a i1 x 1 + . . . + a in x n as a vector product a T i x. Thus, non-strict inequalities are represented as a T i x ≤ b i , strict inequalities as a T i x < b i , and divisibility constraints as d i | a T i xb i . The vector representation is useful whenever we are using the same n variables and do not need to focus on one specific variable. We use it mainly in Chapters 4, 5, and 6.

In contrast, the focused representation is useful when the set of variables changes over time or when we focus on one specific variable. We use it mainly in Chapter 3 because the algorithms in this chapter are related to quantifier and variable elimination. For the focused representation, we select one variable x j as our focus and separate it from the sum of variables. Then we abbreviate the sum over the remaining variables together with the constraint bound as a linear polynomial

p ij := n k=1,k =j a ik x k -b i .
Thus, non-strict inequalities are represented as a ij x j + p ij ≤ 0, strict inequalities as a ij x j + p ij < 0, and divisibility constraints as

d i | a ij x j + p ij .
We also always assume without loss of generality that the coefficient a ij of the focused variable in d i | a ij x j + p ij is non-negative. We can do so because divisibility is sign invariant, i.e.,

d i | a ij x j + p ij ≡ d i | -a ij x j -p ij .
Note that the linear polynomial p ij may still contain variables with negative coefficients.

Integer Coefficients and Bounds

Some of our algorithms for linear integer arithmetic (i.e., CutSat, Cut-Sat++, and CutSat g ) require that all coefficients and constraint bounds are integer values instead of the more general rational values. This restriction is easy to fulfill with the help of the following equivalences: First of all, we make all coefficients a ij = c ij q ij of a constraint I integral by multiplying I with q = lcm(q i1 , . . . , q in ), i.e., the lowest common denominator of the coefficients. This works because of the following equivalences for every positive q ∈ Z [100, 128]:

a T i x ≤ b i ≡ q•a T i x ≤ q•b i , a T i x < b i ≡ q•a T i x < q•b i , and d i | a T i x -b i ≡ q • d i | q • a T i x -q • b i .
Now that both the variables x and the coefficients a i are integers, we can also make the constraint bounds b i integral. For inequalities, it is enough to round the constraint bounds in the right direction, i.e.,

a T i x ≤ b i ≡ a T i x ≤ b i and a T i x < b i ≡ a T i x ≤ b i -1.
Divisibility constraints are simply false if the constraint bound is not integral. So we can just replace them with a trivially false constraint, e.g., 2 | 1.

Highlighting Bounds and Equalities

We also use special representations when we want to highlight that a constraint fulfills some specific properties. For instance, if an inequality is a variable bound and we want to highlight this, then we represent it (depending on the variable bound) as x j ≤ b i for a non-strict upper bound , x j < b i for a strict upper bound , x j ≥ b i for a non-strict lower bound , or x j > b i for a strict lower bound . Generally, our standard inequalities a T i x ≤ b i are represented as upper bounds, i.e., b i is the upper bound of a T i x. A lower bound b i for a T i x has to be negated in our standard representation, so written as -a T i x ≤ -b i . If we, however, want to highlight that a T i x has a lower bound b i , then we can also represent -a T i x ≤ -b i as b i ≤ a T i x to make this fact clearer. Similarly, we can highlight the existence of both a lower bound l i and an upper bound u i for a T i x by writing l i ≤ a T i x ≤ u i . Finally, we can highlight that a T i x has the same upper and lower bound b i -so C contains the two inequalities a T i x ≤ b i and -a T i x ≤ -b i -by stating that C contains the equality a T i x = b i . This also means that an equality is not really a new type of constraint but just syntactic sugar for the set of constraints

{a T i x ≤ b i , -a T i x ≤ -b i }.

Assignments and Solutions

The variables x = (x 1 , . . . , x n ) T in a problem C can be assigned (rational) values s = (s 1 , . . . , s n ) T ∈ Q n . This means we define assignments independent of the variable types. Under each such assignment s, a problem C (or a single constraint I) evaluates to a truth value, so either to true or false. We compute the truth value under the assignment s with the function eval(C, s): eval(C, s)

:= I∈C eval(I, s) eval(a T i x ≤ b i , s) := a T i s ≤ b i eval(a T i x < b i , s) := a T i s < b i eval(d i | a T i x -b i , s) := d i | a T i s -b i
This means that (i) the evaluation happens independent of the variable types; (ii) a constraint system evaluates to true if and only if all its constraints evaluate to true; (iii) a non-strict inequality a T i x ≤ b i evaluates to true if and only if the constant term a T i s simplifies to a rational value s that is less than or equal to b i ; (iv) a strict inequality a T i x < b i evaluates to true if and only if the constant term a T i s simplifies to a rational value s that is less than b i ; and (v) a divisibility constraint d i | a T i xb i evaluates to true if and only if the constant term a T i sb i simplifies to an integer value s that is divisible by d i .

If a constraint (system) evaluates to true under a given assignment s, then we also say that s satisfies the constraint (system). Moreover, we typically abbreviate eval(C, s) by writing C(s). This means that a constraint system C is not only a set of constraints and the conjunction over the same constraints but also a function from Q n to {true, f alse}.

Based on these satisfying assignments, we define solutions to constraint systems as follows:

Definition 2.2.1 ((Mixed) Solutions). A (mixed) solution is a point s ∈ (Q n 1 × Z n 2
) that satisfies all constraints in C and also the types of all variables in C. Moreover, we denote by

M(C) = {s ∈ (Q n 1 × Z n 2 ) : C(s)} the set of mixed solutions to the problem C.
Finding such a mixed solution is the main goal of our linear arithmetic decision procedures. However, there are also other satisfying assignments that are categorized as solutions by the literature [START_REF] Korte | Combinatorial Optimization: Theory and Algorithms[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF]: Definition 2.2.2 (Rational Solutions). A rational solution is a point s ∈ Q n that satisfies all constraints in C, but not the types of the integer variables. Moreover, we denote by Q(C) = {s ∈ Q n : C(s)} the set of rational solutions to the problem C.

Definition 2.2.3 (Integer Solutions

). An integer solution is an integer point s ∈ Z n that satisfies all constraints in C and assigns all variables to integer values. Moreover, we denote by Z(C) = {s ∈ Z n : C(s)} the set of integer solutions to the problem C.

The literature uses these two additional types of solutions because most algorithms that search for a mixed solution do so by solving intermediate constraint systems with relaxed or strengthened variable types. 3 As a result, it became standard to base most definitions in linear arithmetic on the relevant solution type instead of the actual variable types. Variable types are only relevant so we know which variables are supposed to be assigned to integer values in a mixed solution. Now using these solution types we formally define the goal of linear rational/integer/mixed arithmetic independent of variable types: the goal of linear rational/integer/mixed arithmetic is to prove that a given input problem is rational/integer/mixed satisfiable, i.e., has a rational/integer/mixed solution, or that it is rational/integer/mixed unsatisfiable, i.e., the constraints build a conflict and have no rational/integer/mixed solution 4 . We specify at the beginning of every chapter (in the chapter specific preliminaries) the theory we are looking at and the type of solutions/satisfiability we are looking for. In this chapter, we are looking at the theory of linear mixed arithmetic and we abbreviate with C is (un)satisfiable that C is mixed (un)satisfiable.

Finding an integer or mixed solution for any constraint system is an NPcomplete task [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]. Finding a rational solution for a system of inequalities, i.e., a constraint system without divisibility constraints, is much easier and can be done in polynomial time [START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Khachiyan | Polynomial algorithms in linear programming[END_REF]. However, finding a rational solution is again an NP-complete task if our constraint system contains divisibility constraints. The reason is that we can restrict our rational solutions to integer assignments by adding constraints 1 | x j for all variables x j .

Equivalent and Equisatisfiable

Now that we have defined the solutions to our constraints systems, we can also define when two constraint systems are equivalent. Two constraints systems are equivalent if they have the same solutions. Since we have three different types of solutions, this means that we also need three different types of equivalence.

Definition 2.2.4 (Equivalences). Let C 1 and C 2 be constraint systems. C 1 is (rationally) equivalent to C 2 if Q(C 1 ) = Q(C 2 ). C 1 is mixed equivalent to C 2 if M(C 1 ) = M(C 2 ). C 1 is integer equivalent to C 2 if Z(C 1 ) = Z(C 2 ).
Due to the subset relationship of our solution sets, we get the following relationship between our equivalences: C 1 is rationally equivalent to C 2 implies that C 1 is mixed equivalent to C 2 ; and C 1 is mixed equivalent to C 2 implies that C 1 is integer equivalent to C 2 . The reverse is not necessarily true.

We typically talk about equivalent constraint systems when we have to transform our original system into an equivalent system. We need such transformations whenever our original system is too hard, either because it is too hard to solve or because it does not comply to some requirements of our algorithms/theorems.

Obviously, we cannot always find easier equivalent systems. A good alternative is to look for systems that are easier but only equisatisfiable: Definition 2.2.5 (Equisatisfiability). Let C 1 and C 2 be constraint systems. C 1 is rationally equisatisfiable to C 2 if the following two statements are equivalent: C 1 is rationally satisfiable and C 2 is rationally satisfiable. C 1 is mixed equisatisfiable to C 2 if the following two statements are equivalent: C 1 is mixed satisfiable and C 2 is mixed satisfiable. C 1 is integer equisatisfiable to C 2 if the following two statements are equivalent: C 1 is integer satisfiable and C 2 is integer satisfiable.

Equisatisfiable systems are especially useful if there is an efficient way to convert solutions between the two equisatisfiable systems.

Substitutions

Sometimes we do not want to assign all variables x to a fixed value s but just one variable x j to another linear expression p j , which does not contain x j . So we want to replace all occurrences of x j with p j . We call this process substituting x j with p j and denote it by writing C{x j → p j }. We also assume without loss of generality that the substitution C{x j → p j } automatically transforms all of the constraints back into the original representation format.

We are also able to substitute multiple variables x j at once if the substituted variables do not appear in any of the replacement expressions p j . As with the single variable substitution, we denote the substitution over multiple variables as a set σ D,c y,z := {y i → c i + d T i z : i ∈ {1, . . . , n y }}, where y = (y 1 , . . . , y ny ) T and z = (z 1 , . . . , z nz ) T are a partition of the variables in x, D = (d T 1 , . . . , d T ny ) T ∈ Q ny×nz , and c ∈ Q ny . Semantically, Cσ D,c y,z means that each variable y i is substituted with the term c i + d T i z.

Reduction to Non-Strict Inequalities

The common definition of linear arithmetic in the literature supports only one type of constraint: non-strict inequality constraints [START_REF] Boyd | Convex Optimization[END_REF][START_REF] Hillier | Duality Theory and Sensitivity Analysis[END_REF][START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Lemke | The dual method of solving the linear programming problem[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF]. There are multiple reasons for this choice.

First of all, the rational solutions of system of inequalities have some very useful properties. For instance, they define a polyhedron in the ndimensional vector space Q n , where each rational solution is equivalent to a point in the polyhedron [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. In the case that our system C contains only non-strict inequalities, the polyhedron is even closed convex [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. This entails two very useful properties: firstly, the closed convex polyhedron has a surface if it is neither empty nor encompasses the whole Q n ; secondly, any supremum h max = sup{h T x : x ∈ Q n satisfies C} over a linear objective h ∈ Q n is either h max = -∞ because there exists no point satisfying our constraints, h max = ∞ because the supremum is unbounded, or there exists an actual maximum, i.e., there exists an x ∈ Q n that satisfies the constraints and its cost h T x is equal to our supremum h max .

Adding strict inequalities to our constraint system has the effect that our rational solutions are no longer guaranteed to be a closed set. And if we add divisibility constraints, then our rational solutions describe no longer a polyhedron nor are they a convex set. However, these properties are necessary for most classical algorithms and theorems in linear arithmetic. For instance, the classical dual simplex algorithm [START_REF] Schrijver | Theory of linear and integer programming[END_REF] returns only rational solutions on the surface of the polyhedron. It is, therefore, not trivial to adapt all classical algorithms and theorems to also handle strict inequalities or divisibility constraints directly. Instead, we show how to represent both divisibility constraints as well as strict inequalities via non-strict inequalities.

Reducing Divisibility Constraints

We can transform any divisibility constraint and negated divisibility constraint into an equality by introducing additional variables q (and r). For divisibility constraints d i | a T i xb i , this transformation is known as the diophantine representation [START_REF] David | Theorem proving in arithmetic without multiplication[END_REF]:

{d i | a T i x -b i } ≡ ∃q ∈ Z.{d i q -a T i x = -b i },
where q is a fresh integer variable. For negated divisibility constraints d i a T i xb i , there exists a similar transformation:

{d i a T i x -b i } ≡ ∃q ∈ Z, r ∈ Q.{d i q + r -a T i x = -b i , 0 < r < d},
where q is a fresh integer variable and r a fresh rational variable. Both of these transformations describe the formal definition of dividing a T i xb i by d i , i.e., a T i xb i = d i q + r, where q is the quotient of the division and r the remainder. Since the divisibility constraint enforces that d i divides a T i xb i , the remainder r must be zero. Likewise, the negated divisibility constraint enforces that d i does not divide a T i xb i . Therefore, the remainder r lies in the open interval (0, d).

It is actually a big disadvantage that we have to introduce additional variables q (and r) whenever we want to get rid of a (negated) divisibility constraint because divisibility constraints are typically introduced to eliminate variables. For instance, quantifier elimination techniques for linear integer arithmetic introduce divisibility constraints in order to eliminate a quantified variable (see also Chapter 3.4 and [44]). So eliminating the newly introduced divisibility constraint by adding another quantified variable would be counter productive.

Another disadvantage is that the above reduction is only mixed equisatisfiable and not rationally equisatisfiable. For instance, 1 | x j guarantees that x j is assigned to an integer value in all rational solutions and the diophantine representation q = x j does not. This is due to the fact that the transformation moves the integer guarantee out of the evaluation semantics of divisibility constraints and into the type condition of the new variable q.

These disadvantages lead us to the following consequences: (i) we never eliminate divisibility constraints introduced during one of our algorithms and (ii) we are careful to differentiate between the rational relaxation of the original problem and the rational relaxation of the transformed problem.

Reducing Strict Inequalities

We model strict inequalities as non-strict inequalities by generalizing the field Q for our inequality bounds b i and our variable assignments to Q δ [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF].

Lemma 2.3.1 (δ-Rationals [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]). Let a i ∈ Q n and b i ∈ Q. Then a set of linear arithmetic constraints C that contains strict inequalities

C ⊇ C = {a T 1 x < b 1 , . . . , a T m x < b m } is rationally satisfiable iff there exists a rational number δ > 0 such that C δ = (C ∪ C δ ) \ C is rationally satisfiable for all δ with 0 < δ ≤ δ, where C δ = {a T 1 x ≤ b 1 -δ , . . . , a T m x ≤ b m -δ }.
We express the above observation symbolically as an infinitesimal parameter δ > 0. This leads to the ordered vector space Q δ = Q × Q, which we call the δ-rationals, that has pairs of rationals as elements (p, q) ∈ Q δ representing p + qδ with the following operations:

(p 1 , q 1 ) + (p 2 , q 2 ) ≡ (p 1 + p 2 , q 1 + q 2 ) a • (p 1 , q 1 ) ≡ (a • p 1 , a • q 1 ) (p 1 , q 1 ) ≤ (p 2 , q 2 ) ≡ (p 1 < p 2 ) ∨ (p 1 = p 2 ∧ q 1 ≤ q 2 ) (p 1 , q 1 ) < (p 2 , q 2 ) ≡ (p 1 < p 2 ) ∨ (p 1 = p 2 ∧ q 1 < q 2 ) d 1 | (p 1 , q 1 ) ≡ (q 1 = 0) ∧ (d 1 | p 1 ) (p 1 , q 1 ) ≡      p 1 for p 1 ∈ Z p 1 -1 for p 1 ∈ Z and q 1 < 0 p 1 else (p 1 , q 1 ) ≡      p 1 for p 1 ∈ Z p 1 + 1 for p 1 ∈ Z and q 1 > 0 p 1 else (p 1 , q 1 ) ≡ (p 1 + 0.5, q 1 ) |(p 1 , q 1 )| ≡ (|p 1 |, |q 1 |) p + qδ d ≡ ( p d , q d ) dist d (b, c) ≡ b -c d where p i , q i ∈ Q, p = (p 1 , . . . , p n ) T , q = (q 1 , . . . , q n ) T , a ∈ Q, d 1 ∈ Z with d 1 > 0, d ∈ Z ∪ {∞} with d > 0, and b, c ∈ Q n δ [57] 5 .
Given the δ-rationals and Lemma 2.3.1, we can now represent

a T i x < b i by a T i x ≤ b i -δ, where a i ∈ Q n and b i ∈ Q.
i ∈ Q δ . If the bound b i is non-strict, i.e., b i = (p i , 0), then the strict version is b δ i := (p i , -1)
. Otherwise, the bound b i is already strict, i.e., b i = (p i , q i ) with q i < 0, and we just standardize the δ-coefficient to -1, i.e, b δ i := (p i , -1). Furthermore, we abbreviate with bi that we swap between the strict and non-strict version. If b i is non-strict (i.e., b i = (p i , 0)), then bi := b δ i = (p i , -1) gives us the strict version b i . If b i is strict (i.e., b i = (p i , q i ) with q i < 0), then bi := (p i , 0) gives us the non-strict version of b i .

As mentioned before, we now also let the assignments for our variables range over Q δ . This means we now also have δ-rational solutions and δ-mixed solutions and we denote their solution sets by

Q δ (C) = {s ∈ Q n δ : C(s)} and M δ (C) = {s ∈ (Q n 1 δ × Z n 2 ) : C(s)}.
Similar to rational inequality systems, the solutions of our δ-rational inequalities describe a closed convex polyhedron in the n-dimensional Q n δ and methods like the classical simplex algorithm are again complete.

It is also easy to extract a purely rational/mixed solution s ∈ Q n from a δ-rational/mixed solution s ∈ Q n δ . We just have to choose a small enough value δ ∈ Q and replace the parameter δ with this value (Lemma 2.3.1). Therefore, we call δ-rational solutions and δ-mixed solutions just rational solutions and mixed solutions, respectively. Analogously, we also say rationally/mixed satisfiable, rationally/mixed equivalent, and rationally/mixed equisatisfiable when we are actually talking about the δ-rational/δ-mixed versions of these expressions.

Representing our strict inequalities as non-strict inequalities also allows us to use the second property listed above for closed convex polyhedra, i.e., any supremum h max = sup{h T x : x ∈ Q n δ satisfies C} over a linear objective h ∈ Q n is either h max = -∞, h max = ∞, or there exists an actual maximum and not just a limit. This property is especially useful on techniques based on optimization like the largest cube test (see Chapter 4).

There exists an alternative and much easier elimination if all variables of a strict inequality are integer variables. In this case, the equivalence a T i x < b i ≡ a T i x ≤ b i -1 can be used to replace strict inequalities with nonstrict inequalities [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. This version is to be preferred whenever possible as it removes some (δ-)rational solutions that would violate the type of the integer variables.

Standard Input Formats

We previously gave a general definition for constraint systems. This definition not only included non-strict inequalities, but also strict inequalities and divisibility constraints. We need strict inequalities because the negation of a non-strict inequality is a strict inequality, i.e., ¬(a T i x ≤ b i ) ≡ a T i x < b i . This means that a theory solver for linear arithmetic, i.e., a decision procedure for conjunctions of linear arithmetic literals, has to handle strict inequalities in some ways. Furthermore, we need divisibility constraints because they are introduced during the execution of some of our algorithms (CutSat, CutSat++, and CutSat g ).

In the last section, we also explained that we cannot adept classical linear arithmetic algorithms and theorems directly to strict inequalities and divisibility constraints. To resolve this problem, we also presented reductions that turn our too general constraints into non-strict inequalities. This allows us to define our standard input problems-or for short problems-as systems of non-strict inequalities, which we typically abbreviate as systems of inequalities or inequality systems. 6 Thereby, we are also conforming to classical linear arithmetic [START_REF] Boyd | Convex Optimization[END_REF][START_REF] Hillier | Duality Theory and Sensitivity Analysis[END_REF][START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Lemke | The dual method of solving the linear programming problem[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF].

Later in this section, we also present an alternative input format called the tableau representation. We switch to this format whenever we are discussing algorithms in the context of SMT solvers because the tableau representation matches the actual representation inside most SMT implementations [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF][START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF].

Systems of Inequalities

A system of inequalities is a set of inequalities {a T 1 x ≤ b 1 , . . . , a T m x ≤ b m }, which we typically abbreviate as Ax ≤ b [START_REF] Korte | Combinatorial Optimization: Theory and Algorithms[END_REF]. The row coefficients are given by A = (a 1 , . . . , a m ) T ∈ Q m×n , the variables are given by x = (x 1 , . . . , x n ) T , and the inequality bounds are given by b

= (b 1 , . . . , b m ) T ∈ Q m δ .
Moreover, we assume that any constant rows a i = 0 n were eliminated from our system during an implicit preprocessing step. This is a trivial task and eliminates some unnecessarily complicated corner cases.

Since Ax ≤ b and A x ≤ b are just sets, we can write their combination as (Ax ≤ b) ∪ (A x ≤ b ). A special system of inequalities is a system of equations Dx = c, which is equivalent to the combined system of inequalities (Dx ≤ c)∪(-Dx ≤ -c). For such a system of equalities, the row coefficients are given by D = (d 1 , . . . , d m ) T ∈ Q m×n , the variables are given by x = (x 1 , . . . , x n ) T , and the equality bounds are given by c = (c 1 , . . . , c m ) T ∈ Q m . The δ-coefficients q i in the bounds b i = p i + q i δ can take on any value in Q δ . If q i = 0, then the inequality a T i x ≤ b i is equivalent to the non-strict inequality a T i x ≤ p i . If q i < 0, then the inequality a T i x ≤ b i is equivalent to the strict inequality a T i x < p i . If q i > 0, then we have no clear interpretation over the actual rationals (compare also Lemma 2.3.1). For instance, the two inequalities x 1 ≤ δ and -x 1 ≤ -δ describe a rationally satisfiable system of constraints in Q δ , but there is no clear way of interpreting x 1 ≤ δ in Q. Beware also that some of our methods (e.g., the linear cube transformation) can introduce positive δ-coefficients in the bounds. But since we derive all our methods with a semantically clear construction, the semantic interpretation over the rationals is still discernible if the original system has only non-positive δ-coefficients in its inequality bounds before the transformation.

Tableau Representation

We defined our standard input problems as systems of inequalities Ax ≤ b. We do so because most theorems in the literature as well as our own theorems can be proven more intuitively with inequalities. There are, however, some algorithms, e.g., the dual simplex algorithm we present in Section 2.7, for which we prefer a different representation of our input constraints: the tableau representation [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]. In the tableau representation, we partition our variables into two sets: the set of non-basic variables z 1 , . . . , z n ∈ N and the set of basic variables y 1 , . . . , y m ∈ B. The constraints of the tableau representation are then defined as: the so-called tableau Az = y and a set of bounds for the variables L(x j ) ≤ x j ≤ U(x j ) (for x j ∈ N ∪ B). The tableau representation also features two functions L : B ∪ N → Q δ ∪ {-∞} and U : B ∪ N → Q δ ∪ {∞} that map the variables x i ∈ B ∪ N to their upper and lower bound values, respectively. The lower bound value L(x j ) is -∞ for variable x j if x j has no (explicit) lower bound. Similarly, the upper bound value U(x j ) is ∞ for variable x j if x j has no (explicit) upper bound.

We can easily transform a system of inequalities Ax ≤ b into tableau representation by introducing a so-called slack variable s i for every inequality in our system. The type of the slack variable is typically rational, but can be set to integer if its row coefficients a i and all variables with non-zero coefficients are also integers. Our system is then defined by the equalities Az = y (with z := x and y := s) and the bound values L(x j ) := -∞ and U(x j ) := ∞ for every original variable x j and the bound values L(s i ) := -∞ and U(s i ) := b i for every slack variable introduced for the inequality a T i x ≤ b i . So initially, our original variables are the non-basic variables and the slack variables are the basic variables.

We can even reduce the number of slack variables if we transform inequalities of the form a ij • x j ≤ b i directly into bounds for x j . Moreover, we can use the same slack variable for multiple inequalities as long as the left side of the inequality is similar enough. For example, the inequalities a T i x ≤ b i and -a T i x ≤ c i can be transformed into the equality a T i x = s i and the bound values L(s i ) := -c i and U(s i ) := b i .

SMT solvers typically assign the slack variables during a preprocessing step with a normalization procedure based on a variable ordering. After the normalization, all terms are represented in one directed acyclic graph (DAG) so that all equivalent terms are assigned to the same node and, thereby, to the same slack variable. For more details on these simplifications we refer to [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF].

Implied Constraints and Linear Combinations

We say that a constraint system C implies or entails a (set of) constraint(s) I if I evaluates to true for all s ∈ Q δ (C) [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. 7 We also denote this relationship by writing C I. Although equalities and other highlighted constraints are not part of our standard constraints, we still say that a constraint system C implies a highlighted constraint if its equivalent standard representation evaluates to true for all s ∈ Q δ (C). For instance, C h T x = g if and only if {h T x ≤ g, -h T x ≤ -g} evaluates to true for all s ∈ Q δ (C). A constraint I implied by C is explicit if it does appear in C, i.e., I ∈ C. A highlighted constraint I implied by C is also called explicit if its equivalent standard representation C appears in C, i.e., C ⊆ C. Otherwise, implied (highlighted) constraints are called implicit.

Besides our general/rational entailments C I over the rationals, we also use mixed entailments and integer entailments denoted by C M I and C Z I, respectively. And similarly to our general entailment definition,

C M I if I evaluates to true for all s ∈ M δ (C) and C Z I if I evaluates to true for all s ∈ Z(C). Since Z(C) ⊆ M δ (C) ⊆ Q δ (C),
this also means that all generally entailed constraints are also mixed entailed and all mixed entailed also integer entailed, i.e., if C I, then C M I and, if C M I, then C Z I. The reverse is not true as shown by the following example:

{2x 1 + 4x 2 ≤ 7} Z x 1 + 2x 2 ≤ 7 2 and {2x 1 + 4x 2 ≤ 7} x 1 + 2x 2 ≤ 7 2
because dividing a constraint by a positive constant factor does not change its solutions. However, we also know that {2x 1 + 4x 2 ≤ 7} Z x 1 + 2x 2 ≤ 3 because the term x 1 + 2x 2 evaluates under every integer assignment to an integer value so the fractional bound of x 1 + 2x 2 ≤ 7 2 can be rounded down without losing any integer solutions. In contrast, {2x 1 + 4x 2 ≤ 7} does not generally entail x 1 + 2x 2 ≤ 3 because the assignment ( 72 , 0) T is a rational solution for 2x 1 + 4x 2 ≤ 7 but not for x 1 + 2x 2 ≤ 3.

There are types of constraint systems for which we can constructively define their implied/entailed constraints. For instance, any constraint system without rational solutions entails all constraints. Similarly, any constraint system without mixed solutions mixed entails all constraints and integer solutions integer entails all constraints.

Corollary 2.5.1 (Unsatisfiable Entailments). Let I be any linear arithmetic

constraint. If Q δ (C) = ∅, then C I. If M δ (C) = ∅, then C M I. If Z(C) = ∅, then C Z I.
The implied/entailed inequalities of a systems of inequalities Ax ≤ b can also be defined constructively. Either Ax ≤ b has no rational solution and implies all constraints (see Corollary 2.5.1) or all implied inequalities are so-called linear combinations of the inequalities in Ax ≤ b. A linear combination of the inequalities in Ax ≤ b is an inequality h T x ≤ g that resulted from multiplying each inequality in Ax ≤ b by a non-negative factor and adding them up. Formally this means that there exists a vector y ∈ Q m with y ≥ 0 m , y T A = h T , and y T b ≤ g. It is relatively easy to prove that any linear combination of Ax ≤ b is also implied by Ax ≤ b: Corollary 2.5.2 (Linear Combination Corollary [START_REF] Schrijver | Theory of linear and integer programming[END_REF]). Let there exists a y ∈ Q m with y ≥ 0 m , y T A = h T , and y T b ≤ g, i.e., there exists a linear combination of inequalities in Ax ≤ b that results in the inequality h T x ≤ g.

Then Ax ≤ b implies h T x ≤ g. Proof. If s ∈ Q δ (Ax ≤ b), then all a T i s ≤ b i are true. Since all a T i s ≤ b i are constant, we also know that h T s = y T As ≤ y T b ≤ g is true. Hence, every s ∈ Q δ (Ax ≤ b) is also a solution of h T x ≤ g. So Ax ≤ b implies h T x ≤ g.
Before we can finish the proof that all implications are linear combinations, we need the help of the following very important and famous lemma: Lemma 2.5.3 (Farkas' Lemma [START_REF] Farkas | Theorie der einfachen Ungleichungen[END_REF][START_REF] Matoušek | Duality of linear programming[END_REF]). Q(Ax ≤ b) = ∅ iff there exists a y ∈ Q m with y ≥ 0 m and y T A = (0 n ) T so that y T b < 0, i.e., there exists a non-negative linear combination of inequalities in Ax ≤ b that results in an inequality y T Ax ≤ y T b that is constant and unsatisfiable.

Farkas' Lemma was originally formulated over the standard rationals and not the δ-rationals. So we have to prove that Farkas' Lemma still works with δ-rationals:

Lemma 2.5.4 (Farkas' Lemma for Q δ ). Q δ (Ax ≤ b) = ∅ iff
there exists a y ∈ Q m with y ≥ 0 m and y T A = (0 n ) T so that y T b < 0, i.e., there exists a non-negative linear combination of inequalities in Ax ≤ b that results in an inequality y T Ax ≤ y T b that is constant and unsatisfiable. If such a y exists, then we call it a certificate of unsatisfiability or alternatively a conflict (explanation).

Proof. Let us first consider the case where Ax ≤ b is rationally unsatisfiable. Dutertre and de Moura's version of the dual simplex algorithm is a complete and correct algorithm for determining the satisfiability of a linear arithmetic problem over the δ-rationals (for more details see Section 2.7 and [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]). In case the problem is rationally unsatisfiable, the algorithm returns a conflict explanation, which can be turned, together with the final simplex tableau, into the linear combination y ∈ Q m we are looking for. Let us now consider the case where s ∈ Q n δ is a solution for Ax ≤ b. Due to Corollary 2.5.2, we know that s is also a solution to any linear combination y T A ≤ y T b of inequalities in Ax ≤ b. So there cannot exist a y ∈ Q m with y ≥ 0 m , y T A = (0 n ) T , and y T b < 0 because 0 = (0 n ) T s = y T As ≤ y T b < 0 is a contradiction. Now given the δ-rational version of Farkas' Lemma, we can prove that either Ax ≤ b has no rational solution or all implied inequalities are linear combinations of the inequalities in Ax ≤ b:

Lemma 2.5.5 (Linear Implication Lemma [128] 8 ). Let Q δ (Ax ≤ b) = ∅, h ∈ Q n \ {0 n }, and g ∈ Q δ . Then Ax ≤ b implies h T x ≤ g iff
there exists a y ∈ Q m with y ≥ 0 m and y T A = h T so that y T b ≤ g, i.e., there exists a non-negative linear combination of inequalities in Ax ≤ b that results in the inequality h T x ≤ g.

Proof.

Let us first consider the case where Ax ≤ b implies the inequality h T x ≤ g. In this case, the system Ax ≤ b ∪ {-h T x ≤ (-g)} is rationally unsatisfiable. It follows by Lemma 2.5.4 that there exists a (i) y ≥ 0 m and a y m+1 ≥ 0 such that (ii) y T Ay m+1 h = 0 n and (iii) y T b + y m+1 (-g) < 0. It also follows by Lemma 2.5.4 and Ax ≤ b being rationally satisfiable that y m+1 > 0. Now we choose the linear combination y = 1 y m+1 y. This linear combination is positive because of (i). Moreover, it holds that y T A = h T because of (ii). Finally, it holds that y T b < g because of (iii), which implies that y T b ≤ g. Hence, y = 1 y m+1 y is the linear combination that gets us h T x ≤ g.

Let us now consider the case where there exists a y ∈ Q m with y ≥ 0 m and y

T A = h T so that y T b ≤ g. If s ∈ Q δ (Ax ≤ b), then s also satisfies h T x ≤ g since h T x = y T Ax ≤ y T b ≤ g. Hence, Ax ≤ b h T x ≤ g.
Linear combinations are not just useful for their relationship to implied inequalities. We can also use them to prove many more theorems over systems of linear inequalities. For instance, we can prove some properties for minimal sets of unsatisfiable inequalities.

Minimal Sets of Unsatisfiable Inequalities

We call an unsatisfiable set C of inequalities minimal if every proper subset C ⊂ C is satisfiable. Whenever a polyhedron Ax ≤ b is rationally unsatisfiable (i.e., Q δ (Ax ≤ b) = ∅), then there exists a minimal set C of rationally unsatisfiable inequalities so that every inequality in C appears also in Ax ≤ b [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]. We call such a minimal set C a minimal conflict (explanation) for Ax ≤ b's rational unsatisfiability. If we are investigating a minimal set of unsatisfiable inequalities, then we can strengthen Farkas' Lemma as follows:

Lemma 2.5.6 (Farkas' Lemma for Explanations). Let the system of inequalities C = {a T i x ≤ b i : 1 ≤ i ≤ m} be a minimal set of rationally unsatisfiable inequalities. Let A = (a 1 , . . . , a m ) T and b = (b 1 , . . . , b m ) T . Then it holds for every y ∈ Q m with y ≥ 0 m , y T A = (0 n ) T , and y T b < 0 that y i > 0 for all i ∈ {1, . . . , m}.

Proof. Suppose to the contrary that there exists a y ≥ 0 m with y T A = (0 n ) T and y T b < 0 such that one component of y is zero. Without loss of generality we assume that

y m = 0. Let C = {a T i x ≤ b i : 1 ≤ i ≤ m -1}, A = (a 1 , . . . , a m-1 ) T , b = (b 1 , . . . , b m-1 )
T , and y = (y 1 , . . . , y m-1 ) T . Then, y ≥ 0 m-1 , y T A = (0 n ) T , and y T b < 0. However, by Lemma 2.5.4, this implies that (A x ≤ b ) ⊂ C is rationally unsatisfiable. Therefore, C is not minimal, which contradicts our initial assumptions.

CDCL(T): A Framework for SMT Solvers

In this thesis, we present new linear arithmetic techniques for SMT (satisfiability modulo theories) solving. To be more precise, the techniques that we present in Chapters 3-6 are meant to be integrated inside a theory solver for CDCL(T) [START_REF] Ganzinger | DPLL(T): fast decision procedures[END_REF], which is a framework used by most SMT solvers [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF]. 9CDCL(T) [START_REF] Ganzinger | DPLL(T): fast decision procedures[END_REF], also called DPLL(T), is a very general framework that describes a set of interactions between a CDCL-based (conflict-driven-clauselearning-based) SAT solver [START_REF] Zhang | Efficient conflict driven learning in a Boolean satisfiability solver[END_REF] and a theory solver for a given theory T [START_REF] Sebastiani | Lazy satisability modulo theories[END_REF]. 10 If a SAT solver and a theory solver are combined based on these interactions, then they become a decision procedure for ground formulas in clause normal form over the given theory T . Originally, these interactions were described through an interface for the theory solver and included four operations: (i) assert a literal, (ii) check currently asserted literals for theory satisfiability, (iii) return conflict explanation, and (iv) backtrack.

These interactions are used in CDCL(T) as follows: CDCL(T) first creates a propositional abstraction of the input formula, i.e., it replaces the theory atoms with fresh propositional variables. (A map from propositional variables to theory atoms is created as part of this abstraction.) As its se-cond step, CDCL(T) uses a CDCL-based SAT solver to find a propositional model that satisfies the abstracted formula. Next, CDCL(T) checks whether the propositional model is also theory satisfiability, i.e., CDCL(T) asserts in its theory solver the theory literals corresponding to the propositional model and checks them for theory satisfiability. If the propositional model is theory satisfiable, then the overall problem is theory satisfiable and CDCL(T) can stop the search. If the theory solver finds a conflict between the asserted literals, then it returns a conflict explanation. The SAT solver uses the conflict explanation for a conflict analysis that determines a good point for backtracking. Then the SAT solver goes back to the second step and selects a different propositional model that satisfies the abstracted formula. The problem is unsatisfiable if the SAT solver cannot find another propositional model that satisfies the abstracted formula.

There also have been several papers since CDCL(T) was first presented that extend the set of interactions [START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF][START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF][START_REF] Nieuwenhuis | DPLL(T) with exhaustive theory propagation and its application to difference logic[END_REF][START_REF] Nieuwenhuis | Solving SAT and SAT modulo theories: From an abstract Davis-Putnam-Logemann-Loveland procedure to DPLL(T )[END_REF]. The most prominent examples are (i) theory propagation [START_REF] Sebastiani | Lazy satisability modulo theories[END_REF], i.e., propagating literals based on theory reasoning; (ii) theory learning [START_REF] Sebastiani | Lazy satisability modulo theories[END_REF], i.e., using theory reasoning to find and learn clauses implied by the input formula; and (iii) (weakened) early pruning [START_REF] Sebastiani | Lazy satisability modulo theories[END_REF], i.e., checking intermediate propositional models for theory satisfiability in order to find conflicts earlier in the SAT search. These additional interactions may not be necessary for a complete decision procedure, but they have a great impact on practical efficiency.

It is also very important for the practical efficiency of CDCL(T) that the theory solver fulfills certain properties: generation of minimal conflict explanations, high incremental efficiency, and efficient backtracking. We developed our linear arithmetic techniques with these properties in mind.

Propositional Abstraction

The input of CDCL(LA) 11 is a ground linear arithmetic formula in clause normal form, or formally:

F := k C k := k i L ki ,
where F is the whole formula, the C i are the clauses in the formula, and the L ij are the literals in the formula. Each literal L ij is either a linear inequality

a T i x ≤ b i (with b i ∈ Q), a negated linear inequality ¬(a T i x ≤ b i ) (with b i ∈ Q),
a propositional variable p l , or a negated propositional variable ¬(p l ). However, most CDCL(LA) implementations (SPASS-SATT included) use internally a different representation.

Firstly, most SMT theory solvers rely on the tableau representation, which means that they cannot handle linear inequalities directly. We resolve this (as explained in Chapter 2.4.2) through the introduction of slack variables s for all inequalities appearing in our formula F . The result is a tableau Ax = s and a formula F in clause normal form, where all literals are either propositional variables p l , negated propositional variables ¬(p l ), variable bounds

x i ≤ b i or x i ≥ b i (with b i ∈ Q), or negated variable bounds ¬(x i ≤ b i ) or ¬(x i ≥ b i ) (with b i ∈ Q).
Moreover, the combination of tableau Ax = s and formula F is equisatisfiable to the original formula F .

We can also get rid of the negated bounds by using equivalent bounds that rely on δ-rationals (see also Chapter 2.3.2):

¬(x i ≤ b i ) ≡ x i ≥ b i + δ if x i is a rational variable, ¬(x i ≥ b i ) ≡ x i ≤ b i -δ if x i is a rational variable, ¬(x i ≤ b i ) ≡ x i ≥ b i + 1 if x i is an integer variable, ¬(x i ≥ b i ) ≡ x i ≤ b i -1 if
x i is an integer variable. This means we now have a tableau Ax = s and a formula F in clause normal form, where all literals are either propositional variables p l , negated propositional variables ¬(p l ), or variable bounds

x i ≤ b i or x i ≥ b i (with b i ∈ Q δ ).
Next, we abstract our bounds to propositional variables. We do so by replacing each occurrence of a bound x i ≤ b i over a rational variable with a fresh propositional variable p l and each symmetrical occurrence x i ≥ b i + δ with the negated literal ¬(p l ). Analogously, we replace each occurrence of a bound x i ≤ b i over an integer variable with a fresh propositional variable p l and each symmetrical occurrence x i ≥ b i + 1 with the negated literal ¬(p l ). We maintain the connection between propositional variables and abstracted bounds by storing their relationship with the help of a map. The result is an equisatisfiable combination of a tableau Ax = s, a formula F in clause normal form, where all literals are either propositional variables p l or negated propositional variables ¬(p l ), and a function f that maps some of the propositional variables to bounds.

The SAT solver can now run on the propositional CNF formula F and select a model/set of literals. The corresponding bounds to the selected literals are then asserted by the theory solver with the help of the function f . The asserted bounds together with the tableau Ax = s define a linear arithmetic problem in tableau representation. Naturally, all asserted bounds correspond to a (negated) inequality in the original formula F . What might be less obvious is that this set of (negated) inequalities is equisatisfiable to the set of asserted bounds and the tableau Ax = s. Therefore, we are also able to represent all linear arithmetic subproblems of F through our transformed formula.

Standard Arithmetic Decision Procedures for SMT

In this section, we give an overview over the most common linear arithmetic decision procedures used by the SMT community for the CDCL(T) framework. We start with a version of the dual simplex algorithm, which is a decision procedure for linear rational arithmetic. Next we explain bound refinements, a supporting technique that is often used to enhance decision procedures for linear arithmetic. We conclude this section by explaining branch-and-bound, a decision procedure for linear mixed/integer arithmetic, and some of its commonly used extensions.

A Simplex Version for SMT

Most SMT solvers implement the following version of the dual simplex algorithm as their linear rational arithmetic theory solver [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF]. This specific version of the dual simplex algorithm was first presented by Dutertre and de Moura [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF] 12 . Whenever we refer to the simplex algorithm in the remainder of this thesis, we refer to this specific version. It has been proven that almost every variation of the simplex algorithm has an exponential worst-case runtime complexity [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. 13 For the classical simplex algorithm as presented by Dantzig [START_REF] Dantzig | A history of scientific computing[END_REF], this worst case occurs for the KleeMinty cube [START_REF] Klee | How good is the simplex algorithm? Inequalities[END_REF], i.e., a unit hypercube of variable dimension whose corners have been perturbed. Since there are polynomial decision procedures for linear rational arithmetic [START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Khachiyan | Polynomial algorithms in linear programming[END_REF], this also means that there are at least theoretically faster decision procedures. But in contrast to those polynomial decision procedures, the simplex algorithm has all properties necessary for an efficient theory solver: it produces minimal conflict explanations, handles backtracking efficiently, and is highly incremental. In practice, these properties are more important than the difference between polynomial and exponential worst-case complexity [START_REF] Faure | SAT modulo the theory of linear arithmetic: Exact, inexact and commercial solvers[END_REF]. It also helps that the simplex algorithm rarely reaches its worst case in practice.

The input of the simplex algorithm (Figure 2.2) is in tableau representation (Section 2.4.2). Therefore, the input consists of two sets of variables z 1 , . . . , z n ∈ N and y 1 , . . . , y m ∈ B, the two bound value functions L and U, a tableau Az = y, and a set of bounds L(x j ) ≤ x j ≤ U(x j ) for the variables x j ∈ N ∪ B. Each row in this tableau represents one basic variable y i ∈ B: y i = a T i z. This means that the non-basic variables z = (z 1 , . . . , z n ) T define the basic variables y = (y 1 , . . . , y m ) T over the tableau Az = y. The Algorithm 1: pivot(y i , z j ) Input : A basic variable y i and a non-basic variable z j so that a ij is non-zero Effect : Transforms the tableau so y i becomes non-basic and z j basic /* Let y i = a T i z be the row defining the basic variable y i . We rewrite this row as

z j = 1 aij y i -z k ∈N \{zj } a ik aij z k so it defines z j instead */ 1 A ∈ Q m×n ; 2 y i := z j ; 3 for y k ∈ B \ {y i } do y k := y k ; 4 z j := y i ; 5 for z k ∈ N \ {z j } do z k := z k ; 6 a ij := 1 aij ; 7 for z k ∈ N \ {z j } do a ik := -a ik
aij ; /* Then we substitute z j in all other rows with Algorithm 2: update(z j , v)

1 aij y i -k∈N \{zj } a ik aij z k */ 8 for y l ∈ B do 9 for z k ∈ N \ {z j } do a lk := a lk + a lj a ik ;
Input : A non-basic variable z j and a value v ∈ Q δ Effect : Sets the value β(z j ) of z j to v and updates the values of all basic variables

1 for y i ∈ B do β(y i ) := β(y i ) + a ij (v -β(z j )); 2 β(z j ) := v;
Algorithm 3: pivotAndUpdate(y i , z j , v)

Input : A basic variable y i , a non-basic variable z j , and a value v ∈ Q δ Effect : Pivots variables y i and z j and updates the value β(y i ) of y i to v

1 θ := v-β(yi) aij ; 2 β(y i ) := v; β(z j ) := β(z j ) + θ; 3 for y k ∈ B \ {y i } do β(y k ) := β(y k ) + a kj θ; 4 pivot(y i , z j );
Figure 2.1: The pivot and update functions [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF].

simplex algorithm exchanges variables from y i ∈ B and z j ∈ N with the pivot algorithm (Figure 2.1). To do so, we also have to change the tableau via substitution. All tableaux constructed in this way are equivalent to the original tableau Az = y.

Algorithm 4: Check()

Output : Returns true iff there exists a rationally satisfiable assignment for the tableau and the bounds u and l; otherwise, it returns (false,y i ), where y i is the conflicting basic variable

1 while true do 2 select a basic variable y i such that β(y i ) < L(y i ) or β(y i ) > U(y i ) 3
if there is no such y i then return true;

4 if β(y i ) < L(y i ) then 5
select a non-basic variable z j such that 6 (a ij > 0 and β(z j ) < U(z j )) or (a ij < 0 and β(z j ) > L(z j ))

7
if there is no such z j then return (false,y i ) ; (a ij < 0 and β(z j ) < U(z j )) or (a ij > 0 and β(z j ) > L(z j ))

13
if there is no such z j then return (false,y i ) ;

14 pivotAndUpdate(y i , z j , U(y i )) 15 end 16 end Figure 2.2:
The main function of the simplex algorithm [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF] The goal of the simplex algorithm is to find an assignment β that maps every variable x i to a value β(x i ) ∈ Q δ that satisfies our inequality system, i.e., A(β(z)) = β(y) and L(x i ) ≤ β(x i ) ≤ U(x i ) for every variable x i . The algorithm starts with an assignment β that fulfills A(β(z)) = β(y) and L(z j ) ≤ β(z j ) ≤ U(z j ) for every non-basic variable z j ∈ N . Initially, we get such an assignment through our tableau. We simply choose a value L(z j ) ≤ β(z j ) ≤ U(z j ) for every non-basic variable z j ∈ N and define the value of every basic variable y i ∈ B over the tableau: β(y i ) := z j ∈N a ij β(z j ). Note that this only works if we assume that L(x j ) ≤ U(x j ) for every variable x j ∈ N ∪ B, which is easy to guarantee through a small preprocessing step since L(x j ) > U(x j ) already implies that the problem is unsatisfiable. As an invariant, the simplex algorithm continues to fulfill A(β(z)) = β(y) and L(z j ) ≤ β(z j ) ≤ U(z j ) for every non-basic variable z j ∈ N and every intermediate assignment β.

The simplex algorithm finds a rationally satisfiable assignment or an explanation of rational unsatisfiability through the Check() algorithm (Figure 2.2). Since all non-basic variables fulfill their bounds and the tableau guarantees that Az = y, Check() only looks for a basic variable that violates one of its bounds. If all basic variables y i satisfy their bounds, then β is a rationally satisfiable assignment and Check() returns true. If Check() finds a basic variable y i that violates one of its bounds, then it looks for a non-basic Algorithm 5: CheckConflict(y i ) Input : A conflicting basic variable y i , i.e., a variable y i that was returned by Check() in a pair (false, y i ). Output : Returns a minimal set of bounds that cause a conflict with the tableau Az = y variable z j fulfilling the conditions in lines 6 or 12 of Check(). If it finds a non-basic variable z j fulfilling the conditions, then we pivot y i with z j and update our β assignment so β(y i ) is set to the previously violated bound value, which satisfies our invariant once more. If it finds no non-basic variable fulfilling the conditions, then the row of y i and all non-basic variables z j with a ij = 0 build an unresolvable conflict. Hence, Check() has found a row that explains the conflict and it can return unsatisfiable.

1 if β(y i ) < L(y i ) then 2 C := {y i ≥ L(y i )} 3 for z j ∈ N do 4 if a ij > 0 then C := C ∪ {z j ≤ U(z j )}; 5 if a ij < 0 then C := C ∪ {z j ≥ L(z j )}; 6 end 7 end 8 if β(y i ) > U(y i ) then 9 C := {y i ≤ U(y i )} 10 for z j ∈ N do 11 if a ij < 0 then C := C ∪ {z j ≤ U(z j )}; 12 if a ij > 0 then C := C ∪ {z j ≥ L(z j )};
The simplex algorithm terminates due to a variable selection strategy called Bland's rule [START_REF] Robert | New finite pivoting rules for the simplex method[END_REF]. Bland's rule is based on a predetermined variable order and always selects the smallest variables fulfilling the conditions for pivoting according to a predetermined variable order. This already concludes our general description of the simplex algorithm. However, we are still missing the properties that turn the simplex algorithm into a well-suited SMT theory solver, i.e., minimal conflict explanations, high incrementality, and efficient backtracking. We explain why the simplex algorithm fulfills these properties in the remainder of this subsection.

In a typical SMT solver (for more details see Section 2.6), a SAT solver based on CDCL (conflict-driven clause-learning) selects and asserts a set of theory literals that satisfy the boolean model. Then the theory solver verifies that the asserted literals are consistently theory satisfiable. If the theory solver finds a conflict between the asserted literals, then it returns a conflict explanation. The SAT solver uses the conflict explanation to start a conflict analysis that determines a good point for back jumping so it can select a new set of theory literals. Naturally, a good conflict explanation greatly enhances the conflict analysis and, therefore, the remaining search.

The literals asserted in our simplex based theory solver are bounds for our variables. 14 This means that a minimal conflict explanation is a subset C of the asserted variable bounds. In order to be conflicting and minimal, the subset C has to fulfill the following properties: (i) C is, together with the tableau Az = y, unsatisfiable over the rationals; and (ii) any strict subset C ⊂ C is, together with the tableau Az = y, satisfiable over the rationals. Both conditions are also independent of the pivots applied to Az = y because pivoting is an equivalence preserving transformation.

We can derive such a minimal conflict explanation based on the output of the Check() call. If the call to Check() exits in line 7 with (false, y i ), then the conflict explanation is C l = {y i ≥ L(y i )} ∪{z j ≤ U(z j ) : z j ∈ N and a ij > 0} ∪{z j ≥ L(z j ) : z j ∈ N and a ij < 0}. If the call to Check() exits instead in line 13 with (false, y i ), then the conflict explanation is C u = {y i ≥ U(y i )} ∪{z j ≥ L(z j ) : z j ∈ N and a ij > 0} ∪{z j ≤ U(z j ) : z j ∈ N and a ij < 0}. We can compute these sets with the function CheckConflict(y i ) (see Figure 2.3).

The sets are conflicting because C l ∪{a T i z-y i ≤ 0} and C r ∪{y i -a T i z ≤ 0} are minimal conflicts with regard to Farkas' Lemma (Lemma 2.5.6). Note that a T i zy i ≤ 0 and y ia T i z ≤ 0 are the two inequalities that define the i-th row a T i z = y i in the tableau Az = y.

Incremental Extension and Backtracking

Most CDCL(T) solvers do not just check full boolean models for theory satisfiability but also some of the intermediate partial models. This is called (weakened) early pruning [START_REF] Sebastiani | Lazy satisability modulo theories[END_REF]. If the checked model is not theory satisfiable, then the SMT solver uses the conflict (i) to either prove that the whole 14 Actually, the literals we assert are full inequalities a T i x ≤ bi. Due to slacking, the left side of those constraints is abstracted to a slack variable s such that s = a T i x. The definition of the slack variable s = a T i x is directly stored in the simplex tableau and only a bound s ≤ bi remains as the literal for the SMT solver (see Section 2.4.2 for more details).

l i if L(x i ) < l i ≤ U(x i ).
If this violates the simplex invariants, then updates β accordingly.

1 if U(x i ) < l i then return false; 2 if L(x i ) < l i then 3 O := [[O, (x i ≥ L(x i ), d)]] 4 L(x i ) := l i 5 end 6 if l i ≤ β(x i ) then
u i if L(x i ) ≤ u i < U(x i ).
If this violates the simplex invariants, then updates β accordingly.

1 if L(x i ) > u i then return false; 2 if U(x i ) > u i then 3 O := [[O, (x i ≤ U(x i ), d)]] 4 U(x i ) := u i 5 end 6 if u i ≥ β(x i ) then return true; 7 if x i ∈ N then update(x i , u i ); 8 return unknown Figure 2
.4: Incremental assert functions for the simplex algorithm [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF] problem was unsatisfiable or (ii) to find a prefix of the model that is still able to satisfy the conflict. This means that subsequent theory problems are connected as follows: either the problems are incrementally connected , i. Based on this, we call the runtime advantage an algorithm gains from having already solved a subset of the problem its incremental efficiency. Since most problems sent to an SMT theory solver are incrementally connected, its incremental efficiency is a major factor in determining its total efficiency. Similarly, we want to reduce the overhead that our theory solver encounters when we have to backtrack, i.e., remove some of the literals in reverse chronological order. The less overhead this causes the more efficient our theory solver can backtrack. The simplex algorithm is in fact able to both add literals incrementally efficient and to backtrack efficiently.

To this end, we first have to extend the simplex state to include more than just two sets of variables N and B, the two bound value functions L and U, the tableau Az = y, and the current assignment β. We additionally include a sequence of old bounds O, the current backtrack level d, and a backup assignment ω. Initially, O is empty, the current backtrack level d is zero, the bound value functions L and U map all variables to -∞ and ∞, respectively, the backup assignment ω and the current assignment β assign all variables to zero, and the initial tableau Az = y has a slacked row a T i z = y i for all inequalities a T i z ≤ b i that appear in the SMT input problem. The latter is necessary so we never have to extend the tableau.

We extend our bound value functions-and, thereby, the set of literals we consider-with the functions AssertLower() and AssertUpper() (Figure 2.4). Both functions have three possible return values: false, true, and unknown. They return false if the bound we want to add contradicts another bound of the same variable, e.g., we already asserted x i ≤ 5 and are now asserting x i ≥ 6. This is necessary to guarantee our initial assumption that L(x i ) ≤ U(x i ) for all variables x i ∈ N ∪ B. They return true if the new bound value is already satisfied by our current assignment β. And they return unknown otherwise. The newly asserted bound replaces the current bound value if it is stricter, e.g., if we assert x i ≤ u i , then x i ≤ u i is stricter than x i ≤ U(x i ) whenever u i < U(x i ). If we replace a bound value, then we also append the replaced bound to the end of the sequence O together with the current backtrack level. This is necessary for our efficient backtracking algorithm. Both functions also have to maintain the invariant L(z j ) ≤ β(z j ) ≤ U(x i ) for every variable z j ∈ N . This means that we call update(z j , v) whenever the new bound value v violates a non-basic variable z j ∈ N .

Let us now look at a series of assertions that extend our current sequence of literals. 15 First of all, we can assume without loss of generality that we start such a series of assertions from a simplex state with a rationally satisfiable assignment for the current bounds. We can stop our assertions as soon as any of the assertions in the series return false. In this case, we have found a conflict (i.e., the set consisting of the two contradicting bounds) and our model can no longer be satisfiable over the rationals. If all of our assertions return instead true, then our old assignment also satisfies all new bounds. Therefore, the extended model is still satisfiable over the rationals. If the assertions return at least once unknown but never false, then we do not know whether the extended model is satisfiable or unsatisfiable over the rationals. However, all invariants for the simplex algorithm hold. Therefore, we can use Check() to determine the rational satisfiability of our model. These assert functions are so efficient because they maintain the simplex invariants at the cost of a negligible overhead and because they never have to change the tableau. Moreover, any previous pivots to the tableau do not hurt the efficiency of the Check() function. In fact, the previous pivots prevent us from visiting assignments that we already eliminated as unsatisfiable. This means that Check() is more efficient if it previously solved a subset of the problem. Therefore, it is incrementally efficient.

O := O 4 if γ = (x i ≥ l i ) then L(x i ) := l i ; 5 if γ = (x i ≤ u i ) then U(x i ) := u i ; 6 end 7 β := ω 8 d := d
Backtracking, i.e., removing bounds in reverse chronological order, can also be done efficiently with the simplex algorithm. However, we first have to define the points in our chronological order that we want to backtrack to. To this end, let us look again at the goal of a typical CDCL(T) solver.

A typical CDCL(T) solver wants to find a complete boolean model (i.e., sequence of literals) that is also theory satisfiable. Therefore, our goal is to find a sequence of bounds that is satisfiable over the rationals. Since an unsatisfiable sequence of bounds only becomes satisfiable by removing bounds, this also means that we always want to backtrack to satisfiable prefixes of our bound sequence. An SMT solver can mark a subset of these satisfiable prefixes as its backtrack points through the function AddBTPoint() (Figure 2.5).

Note that we always backtrack in a linear order. Therefore, the backtrack points created by AddBTPoint() are actually backtrack levels d and we simply increment our current backtrack level whenever we need a new backtrack point. Apart from that, AddBTPoint() also makes a backup ω of the latest rationally satisfiable assignment. We need only one backup assignment for all backtrack points because ω satisfies per definition a set of bounds C if it satisfies all subsets of C. This means that ω is a rationally satisfiable assignment for all previous backtrack levels.

Thanks to this backup assignment ω and our old bounds sequence O, we are now able to efficiently backtrack to any previous backtrack level d with the function Backtrack(d ) (Figure 2.5). This function is efficient because it only reverts the bounds and exchanges the assignment. So no complex calculations are necessary.

Bound Refinements/Propagations

Bound refinement [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF], also called bound propagation, is a technique for linear arithmetic but in itself not a decision procedure. We use bound refinement in three cases: (i) we use bound refinement as a form of theory propagation in the CDCL(T) framework (for more details see Section 2.6); (ii) we use bound refinement as an extension to the branch-and-bound approach that refines and strengthens rational relaxations (for more details see Subsection 2.7.3); and (iii) we use bound refinement as an arithmetic alternative to unit propagation in the CDCL-like CutSat++ calculus (for more details see Chapter 3).

Bound refinement takes as input an inequality a ij x j +p ij ≤ 0 and several variable bounds from our current set of constraints C. Then it tries to propagate an entailed variable bound for (at least) one of the variables x j with non-zero coefficient a ij . In the case that bound refinement succeeds, it produces a variable bound x j ≤ u j (x j ≥ l j ) that is not subsumed by an existing bound in C. 16 The produced bound x j ≤ u j (x j ≥ l j ) can then be explicitly added to C. The entailed bounds are computed as follows:

Lemma 2.7.1 (Lower Bound Refinement). Let a T i x ≤ b i be an inequality in C. Let x j be a variable with a ij < 0. Let x k ≥ l k be lower bounds in C for all variables x k = x j with a ik > 0. Let x k ≤ u k be upper bounds in C for all variables x k = x j with a ik < 0. Then C (x j ≥ l j ), where

l j := a ik >0,k =j a ik |a ij | • l k + a ik <0,k =j a ik |a ij | • u k -b i |a ij | .
Proof. Follows from Lemma 2.5.5. A lower bound x j ≥ l j is written in standard notation as -x j ≤ -l j . Therefore, C contains the inequalities

{a T i x ≤ b i } ∪ {-x k ≤ -l k : x k = x j and a ik > 0}∪ {x k ≤ u k : x k =
x j and a ik < 0} . This also means that C implies -x j ≤ -l j if there exists a linear combination of those inequalities that results in -x j ≤ -l j . We receive this linear combination by multiplying a T i x ≤ b i with y := 1 |a ij | and the bounds with

y k := |a ik | |a ij |
and by adding them together. This results in

y • a T i x + a ik >0,k =i y k • (-x k ) + a ik <0,k =i y k • x k = k =i a ik |a ij | • x k - a ik >0,k =i a ik |a ij | • x k - a ik <0,k =i a ik |a ij | • x k -x j = -x j on the left side of the combined inequality and b i |a ij | - a ik >0,k =i a ik |a ij | • l k - a ik <0,k =i a ik |a ij | • u k = -l j
on the right side of the combined inequality. Thus C (x j ≥ l j ). Lemma 2.7.2 (Upper Bound Refinement). Let a T i x ≤ b i be an inequality in C. Let x j be a variable with a ij > 0. Let x k ≥ l k be lower bounds in C for all variables x k = x j with a ik > 0. Let x k ≤ u k be upper bounds in C for all variables x k = x j with a ik < 0. Then C (x j ≤ u j ), where

u j := b i |a ij | - a ik >0,k =j a ik |a ij | • l k - a ik <0,k =j a ik |a ij | • u k .
Proof. This proof is analogous to the proof of Lemma 2.7.1.

For both refinement methods, we call the inequality a T i x ≤ b i and the set of bounds C used for the refinement the explanation for the propagated bound. We do so because (a T i x ≤ b i ) ∪ C imply the propagated bound and every subset of (a T i x ≤ b i ) ∪ C does not. One major disadvantage of bound refinement is that it can cause divergence if we propagate new bounds repeatedly without any restrictions:

Example 2.7.3. Let C = {x 1 ≥ 0, x 2 ≥ 0, x 1 -x 2 ≤ 0, -x 1 + x 2 ≤ -1}.
Then we can propagate arbitrarily many lower bounds for x 1 and x 2 that are not subsumed by the previously propagated bounds. We get these bounds by alternately propagating for k = 0, 1, 2, . . .:

• x 1 ≥ k + 1 from -x 1 + x 2 ≤ -1 and x 2 ≥ k; and • x 2 ≥ k + 1 from x 1 -x 2 ≤ 0 and x 1 ≥ k.
As a consequence of these infinite propagation sequences, we always have to restrict bound refinement in some way when we use it in practice. For instance, (i) we only propagate bounds as part of theory propagation that already appear in our input formula; (ii) we set a fixed threshold on bound refinements per branching node in the branch-and-bound approach; and (iii) we use for the CutSat++ calculus a propagation strategy (see Definition 3.5.6) that prevents propagation cycles.

Branch-And-Bound

Branch-and-bound [START_REF] Ailsa | An automatic method for solving discrete programming problems[END_REF][START_REF] John | An algorithm for the traveling salesman problem[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF] is the most common approach to handle linear mixed problems [START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF]. For instance, most theory solvers for linear integer and mixed arithmetic are based on branch-and-bound [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF]. The general idea behind branch-and-bound is to split the original problem into two or more new problems (called branches) that are easier to solve. This is done in the following way: First, we compute a rational solution for our problem C. If there exists none, then we can stop because there also exists no mixed solution. If the rational solution is a mixed solution, then we can also stop because we have found our mixed solution. If the rational solution is not a mixed solution because the assignment s j for the integer variable x j is not an integer value, then we split our problem into two simpler problems C l and C u and solve them recursively. We do so by exploiting the following trivial fact: C has a mixed solution if and only if C l := C ∪ {x j ≥ s j } or C u := C ∪ {x j ≤ s j } has a mixed solution. C l and C u are simpler than C because they have together less rational solutions than

C, i.e., Q δ (C) := Q δ (C l ) Q δ (C r ) Q δ (C ∪ { s j < x j < s j }).
Now that we have discussed the general idea behind branch-and-bound, let us define branch-and-bound more formally with all of the branch-andbound related terminology that we use in this thesis. The main data structure of branch-and-bound is a branching tree and each of the tree's nodes represents one system of inequalities. A branching tree and its nodes are arranged as follows: (i) each tree has exactly one root node; (ii) the root node represents the original system of inequalities C 0 ; (iii) except for the root node, all nodes have exactly one other node declared as their parent;

(iv) the ancestors of a node are the node's parent and the parent node's ancestors; (v) the root node is an ancestor for all other nodes inside the tree; (vi) a node C is either a leaf , i.e., it is not declared as the parent for any node inside the tree, or it has two child nodes C l and C u , i.e., the parent for C l and C u is C; and (vii) if C has child nodes, then C has a mixed solution if and only if one of its children has a mixed solution. Moreover, each node is marked with exactly one of the following labels at a time: (a) a node is marked as pruned if branch-and-bound determines that the node has no mixed solutions; (b) a node is marked as branched if it is the parent of another node; and (v) a leaf is marked as active if it is not pruned and A denotes the set of active nodes. Branch-and-bound works because it maintains the following invariant: the original problem C 0 has a mixed solution if and only if one of the active nodes C ∈ A has a mixed solution. The invariant holds initially because the initial tree consists only of the input problem C 0 as the root node, which is at that point also the only active node. It is maintained during the algorithm thanks to the conditions (i)-(vii) that define branching trees.

As summarized before, the branch-and-bound algorithm works as follows: In every step, one active node C ∈ A is selected and removed from A. Then we use a decision procedure for linear rational arithmetic, such as the simplex algorithm, to determine if C has a rational solution. If C has no rational solution, then it is also mixed unsatisfiable and the node is marked as pruned and we continue to select the next active node. Otherwise, branch-and-bound has found a rational solution β(x) := s ∈ Q δ (C) for C. If it holds that all integer variables x j ∈ Zvars(C) are assigned to integer values β(x j ) ∈ Z, then β(x) := s is also a mixed solution for C and the original problem C 0 . In this case, branch-and-bound stops its search and returns s. If there is an integer variable x j not assigned to an integer value (i.e., β(x j ) ∈ Z), then branch-and-bound selects one such variable as the branching variable and makes a case distinction based on its branching value β(x j ): either it holds that x j ≤ β(x j ) or x j ≥ β(x j ) . (These bounds are also called branching bounds.)

Every such case distinction creates two new branches of the original problem. The two branches are C l := C ∪ {x j ≥ β(x j ) } and C u := C ∪ {x j ≤ β(x j ) } and C has a mixed solution if and only if C l or C u has a mixed solution. Additionally, the new branches C l and C u become new active nodes (A := A ∪ {C l , C u }) with C as their parent. This inductively guarantees that the original problem C 0 has a mixed solution if and only if one of the active nodes C ∈ A has a mixed solution. After adding the new branches to A, branch-and-bound continues recursively and tests the next active node. If A is empty and we cannot select any more active nodes, then the original problem C 0 has no mixed solution.

Extensions to Branch-And-Bound

The branch-and-bound algorithm has many extensions and some of them are necessary for an implementation that is efficient in practice. Here we present the most popular extensions used by the SMT community [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].

Cutting Planes

A cut is any inequality that is mixed implied but not generally implied [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. This means that adding the cut to our inequality system produces a problem with the same mixed solutions but less rational solutions.

There are two advantages of adding cuts to our nodes: (i) branch-andbound is better informed because there are less rational solutions that might mislead it and (ii) some previously rationally satisfiable nodes become rationally unsatisfiable, which means that we can prune the node earlier. The disadvantage is that computing the rational solution becomes more expensive the more constraints we have.

There are many different ways to compute cuts and cuts are typically named after the way they are computed, e.g., gomory cuts [START_REF] Gomory | Outline of an algorithm for integer solutions to linear programs and an algorithm for the mixed integer problem[END_REF]. There even exist procedures that compute finite sequences of cuts that reduce the rational solutions of a problem until the existence of a rational solution implies the existence of a mixed solution. Obviously, these procedures can be easily turned into a complete decision procedure for linear mixed arithmetic. Unfortunately, these procedures are also too slow to help in practice.

Constraint Tightening

Constraint tightening is a very basic inprocessing technique for linear mixed arithmetic [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. Let a T i x ≤ b i be an inequality so that a i ∈ Z n , all a ij = 0 belong to integer variables (i.e., j > n 1 ), and g = gcd{a ij : for j = 1, . . . , n}. 17 Then the tightened version of a T i x ≤ b i is defined as

1 g • a T i x i ≤ b i g .
A tightened constraint contains all mixed solutions of the original constraint but might not contain all rational solutions. This means a constraint tightening that actually changes an inequality is a cut.

Bound Propagation

Bound propagation, which we explained in Section 2.7.2, is also useful and popular as an inprocessing technique for branch-and-bound [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF]. In many cases, bound propagation combined with constraint tightening is enough to produce another cut. These cuts are not very strong but they have the advantage that they always subsume another inequality/bound. Thus, bound propagation does not increase the number of constraints and, therefore, not the time for computing a rational solution. We recommend to always apply (a bounded number of) bound propagation(s) before calculating a rational solution for a new branch.

Rounding Heuristics

Rounding heuristics are an easy way to turn any rational solution β(x) = s computed during branch-and-bound, into an assignment s ∈ Q n 1 δ × Z n 2 that fulfills the types of all variables [START_REF] Fischetti | The feasibility pump[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF]. We simply round all s j with j > n 1 to an integer value s j and keep all other values as before. Beware that this new assignment might very well violate the node or input constraints. We, therefore, have to evaluate whether s is a mixed solution for our original problem C 0 . If the heuristic solution s satisfies the original problem, then we stop our branch-and-bound search and return the solution instead. Otherwise, branch-and-bound ignores s and continues the branchand-bound search with s.

(Rounding) heuristics are easy to determine and to verify. They typically are a shortcut to a mixed solution if branch-and-bound is already close to one. As with cuts, this might reduce the number of branches needed by our branch-and-bound search. The only contrast is that cuts remove mixed unsatisfiable branches and heuristic solutions mixed satisfiable branches.

In the literature, any heuristic is called a rounding heuristic that rounds the values assigned to integer variables to integer values. We do not care whether we round up s j := s j , down s j := s j , or simple s j := s j . In practice, most implementations use simple rounding.

(Un)Bounded and (Un)Guarded

We mentioned before that there exist algorithms that find a rational solution for a system of inequalities Ax ≤ b in polynomial time [START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF][START_REF] Khachiyan | Polynomial algorithms in linear programming[END_REF]. In contrast, the theories of linear integer and linear mixed arithmetic are NPcomplete [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]. So at least for now, all decision procedures for those two theories have an exponential runtime worst-case. Some of those decision procedures (e.g., branch-and-bound) generally perform well in practice. Most of them share, however, one class of problems on which they perform extremely poorly: unbounded problems [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. Definition 2.8.1 (Bounded Direction18 ). A direction/vector h ∈ Q n \{0 n } is bounded in the constraint system C if there exist l, u ∈ Q δ such that C (rationally) implies h T x ≤ u and -h Example 2.8.3. Let us look at two series of problems (see Fig. 2.9):

x 1 ≥ k x 2 ≥ k x 1 ≥ k+1 x 2 ≥ k
C 1 k := {x 1 ≥ k, x 2 ≥ k, 3x 1 -3x 2 ≤ 2, -3x 1 + 3x 2 ≤ -1} and C 2 k := {x 1 ≥ k + 1, x 2 ≥ k, 3x 1 -3x 2 ≤ 2, -3x 1 + 3x 2 ≤ -1}
. In these two series, all problems have rational solutions but no integer solutions. If we perform branch-and-bound on C 1 k to find an integer solution, then we first compute a rational solution for C 1 k , e.g., (x 1 , x 2 ) T = ( 2k+1 3 , k) T . Since x 1 is set to a fractional value, branch-and-bound has to branch on x 1 . This creates the two new problems

C l := C 1 k ∪ {x 1 ≤ k} and C r := C 1 k ∪ {x 1 ≥ k + 1}
on which we now recursively perform branch-and-bound. Since C r is equivalent to C 2 k , one of the recursive calls is equivalent to performing branchand-bound on C 2 k . Again, we first compute a rational solution for C 2 k , e.g., (x 1 , x 2 ) T = (k + 1, 2k+2 3 ) T . Since x 2 is set to a fractional value, branch-andbound has to branch on x 2 . This creates the two new problems

C l := C 2 k ∪ {x 2 ≤ k} and C r := C 2 k ∪ {x 2 ≥ k + 1}
on which we now recursively perform branch-and-bound. However, C r is equivalent to C 1 k+1 , which means one of the recursive calls is equivalent to performing branch-and-bound on C 1 k . This means we always cycle between the two cases described by the two series of problems, i.e.,

C 1 k → C 2 k → C 1 k+1 → C 2 k+1 → .
. ., and branch-and-bound diverges.

Similar diverging examples can be constructed for many other decision procedures.

A Priori Bounds

In contrast to unbounded problems, termination for bounded problems C is quite easy to achieve. Due to the implied bounds, any mixed solution s ∈ M δ (C) must lie between the implied variable bounds l j ≤ x j ≤ u j , i.e., s ∈ M δ (C) implies that l j ≤ s j ≤ u j for j = 1, . . . , n. This means that branch-and-bound branches at most (u n 1 +1l n 1 +1 + 1) • . . . • (u nl n + 1) times before it terminates. Similar arguments can be made for most other decision procedures.

Termination on unbounded problems is much harder. Many decision procedures for linear arithmetic only terminate on unbounded problems because they assume an implicit preprocessing step that transforms every problem into an equisatisfiable bounded problem. And one such class of transformations is called a priori bounds. A priori bounds intersect the original problem with a cube that is so large that the problem only has a mixed solution outside of the cube if it also has one inside the cube. For example, the a priori bounds presented by Papadimitriou [START_REF] Papadimitriou | On the complexity of integer programming[END_REF] guarantee that a problem has a mixed solution if and only if the problem extended by the bounds |x i | ≤ 2n(ma) 2m+1 for every variable x i has a mixed solution. In these a priori bounds, n is the number of variables, m the number of inequalities, and a the largest absolute value of any integer coefficient or constant in the problem. By extending a problem with those a priori bounds, we reduce the search space for a branch-and-bound solver (and many other mixed arithmetic decision procedures) to a finite search space. So branch-and-bound is guaranteed to terminate. However, a priori bounds typically describe a search space that is so large that it cannot be explored in reasonable time. For instance, the a priori bounds for Example 2.8.3 are in the order of 10 billions. So a priori bounds help us in theory but not in practice. As a practically efficient alternative, we present in Chapter 6 an equisatisfiable transformation that also makes branch-and-bound complete. But first we present here some additional facts on (un)bounded directions.

Bounded Basis

A direction h i is explicitly bounded if there exist constants l i , u i ∈ Q δ so that (l i ≤ h T i x ≤ u i ) ∈ C.
They are easy to detect because it is enough to compare all constraints in C to find them. In contrast, unbounded directions and directions h that are implicitly bounded , i.e., bounded directions that are not explicit, are much harder to differentiate. Since there are infinitely many directions, it is not even possible to determine for each direction explicitly whether it is bounded or not. There are, however, finite representations for the bounded directions in a constraint system: Lemma 2.8.4 (Dependent Bounded Directions). Let C be a constraint system and let x = (x 1 , . . . , x n ) T be all variables appearing in C. Let H = (h 1 , . . . , h k ) T be a matrix consisting of bounded directions h 1 , . . . , h k ∈ Q n \ {0 n } in C. Then any linear dependent direction h of h 1 , . . . , h k is also bounded, i.e., any h is also bounded for which there exists a y ∈ Q k with y T H = h T .

Proof. Since the directions h i are bounded, we know that there exist con-

stants l i , u i ∈ Q δ such that C l i ≤ h T i x ≤ u i . If we now combine these bounds into vectors l = (l T 1 , . . . , l T k ) and u = (u T 1 , . . . , u T k ), then C y T l ≤ h T x = y T Hx ≤ y T u because of Corollary 2.5.2. Thus, h is also bounded in C.
The above lemma gives us instructions on how to extract infinitely many bounded directions from a finite set of bounded directions. A finite representation of all bounded directions in C is then any maximum set of linear independent bounded directions in C: Definition 2.8.5 (Bounded Basis). Let C be a constraint system and let x = (x 1 , . . . , x n ) T be all variables appearing in C. Then H = (h 1 , . . . , h k ) T is a bounded basis for C if: (i) all h 1 , . . . , h k ∈ Q n \ {0 n } are linear independent bounded directions in C and (ii) there exists no linear independent direction

h ∈ Q n \ {0 n } to H that is bounded in C.
Corollary 2.8.6 (Bounded Basis). Let C be a constraint system with n variables and H ∈ Q k×n a bounded basis of C. Then any bounded direction h in C is a linear combination of H, i.e., h is bounded iff there exists a

y ∈ Q k with y T H = h T .
The number of variables n in the constraint system is an upper limit for the number of linear independent bounded directions. If there are n linear independent bounded directions, then any direction can be constructed by their basis. Therefore, the whole system is bounded.

Corollary 2.8.7 (Bounded System). Let C be a constraint system with n variables. Then C is bounded if there are n linear independent bounded directions.

The bounded systems that are easiest to recognize are so-called guarded systems, i.e., systems where every variable is explicitly bounded. 20 Due to their explicit boundedness, we sometimes also call them explicitly bounded systems. For the reverse reason, we also sometimes use guarded variables as an alternative name for explicitly bounded variables and unguarded variables for implicitly bounded and unbounded variables.

For unguarded systems, i.e., constraint systems that are not guarded, finding a bounded basis is more complicated. In the best case, we only need to check all explicitly bounded directions for linear independence. In the worst case, we even have to find implicitly bounded directions to complete our basis. Especially, the latter is a non-trivial task.

One of the contributions of this thesis is a method that computes a bounded basis for any system of linear inequalities (see Chapter 5.5). The method has polynomial runtime complexity and we explain how it can be implemented so it is incremental and efficient in practice. Moreover, we explain how to use the method to also derive actual bound values for our bounded basis.

With this method, we are also able to partition all systems of linear inequalities into four cases: guarded systems, unguarded but bounded systems, absolutely unbounded systems, and partially unbounded systems. We already presented the first two cases in the previous paragraphs and together they contain all bounded systems. Naturally, the other two cases contain all unbounded systems.

Types of Unbounded Systems

An absolutely unbounded system is a system where all directions are unbounded (see Figure 2.8 for an example). This also guarantees that it always has mixed/integer solutions: Lemma 2.8.8 (Absolutely Unbounded [START_REF] Bromberger | Fast cube tests for LIA constraint solving[END_REF]). If all directions are unbounded in a system of inequalities Ax ≤ b, then Ax ≤ b has an integer solution and, therefore, also a mixed solution.

In Chapter 4, we present two cube tests that detect and solve constraint systems with infinite lattice width (another name for absolutely unbounded systems [START_REF] Kannan | Covering minima and lattice point free convex bodies[END_REF]) in polynomial time. The case of absolutely unbounded systems is, therefore, trivial and branch-and-bound can be easily extended so it also becomes complete for absolutely unbounded systems.

The actual difficult case occurs when some directions are bounded and others unbounded. We call these systems partially unbounded (see Figure 2.6 for an example). They are problems for which branch-and-bound and most other algorithms diverge or become inefficient in practice. They are also the focus of the bounding transformations that we present in Chapter 6.

Other Geometric Objects

We mentioned before that the rational solutions to a system of inequalities define a geometric object that is called a polyhedron. Decision procedures for linear mixed arithmetic typically use these rational solutions as a guideline to find mixed solutions. These decision procedures also often have the same limitation: they consider only one single rational solution at a time.

An alternative is to consider sets with multiple rational solutions for the exploration of a polyhedron. Such sets of solutions should fulfill two properties to be useful in practice: (i) we must be able to find these sets in polynomial time (or at least an approximation); and (ii) we must be able to determine if the set contains a mixed solution in polynomial time. We actually discovered classes of sets with these properties. Here, we define these sets and later, in Chapter 4, we explain how to find and use them in inequality systems.

d-Norm Balls

The sets we consider in Chapter 4 belong to a class of geometric objects called d-norm balls (see Figure 2.10 for examples) [START_REF] Mahlon | Normed linear spaces[END_REF][START_REF] Huang | Volume of unit ball in an n-dimensional normed space and its asymptotic properties[END_REF]. We define a d-norm ball D d r (z) with radius r ∈ Q δ (r ≥ 0) around the center point z as the set of points y ∈ D d r (z) with at most d-norm distance r to z (i.e., dist d (z, y) ≤ r), or formally:

D d r (z) = {y ∈ Q n δ : dist d (z, y) ≤ r}.
We also say that the points contained in D d r (z) are its rational solutions and its intersection with Z n and Q n 1 δ × Z n 2 its integer and mixed solutions, respectively.

All d-norm balls also fulfill four additional properties that make them practical: they all describe (i) convex sets, (ii) with a center point z, (iii) they are point reflection symmetrical to z, and (iv) they are reflection symmetrical to the axis-parallel hyperplanes going through z.

The definition over the distance also guarantees that a d-norm ball D The problem is that dist d (., .) returns the dth root of a rational number, so not necessarily a rational number (with the exception of d = 1 and d = ∞). However, we do not actually have to compute dist d (z, y) to evaluate dist d (z, y) ≤ r. Instead we can evaluate the equivalent statement

n j=1 |z j -y j | d ≤ r d .
Despite the exponents, we can compute and save these numbers in polynomial time and space if we encode them in binary representation. Thus, we are able to determine in polynomial time whether a given d-norm ball has a mixed solution.

(Axis-Parallel) Hypercubes

Most parts of Chapter 4 do not focus on general d-norm balls but only on ∞-norm balls. An ∞-norm ball is a hypercube that is parallel to the coordinate axes [START_REF] Mahlon | Normed linear spaces[END_REF][START_REF] Huang | Volume of unit ball in an n-dimensional normed space and its asymptotic properties[END_REF]. For simplicity, we call these restricted hypercubes cubes. Cubes have two major advantages over most other d-norm balls 21 : (i) the norm they are based on contains neither roots nor exponents and (ii) they are polyhedra, which means that we can model them through systems of inequalities. The equivalent system of inequalities for a cube with edge length e ∈ Q δ (with e ≥ 0) and center z ∈ Q n δ is {z j -e 2 ≤ x j ≤ z j + e 2 : j = 1, . . . , n}. However, our methods do not actually require the constraint definition of the cubes but just the linearity it implies.

When we are talking about cubes, we are also typically using the edge length e instead of the radius r = e 2 to measure the size of the cube. For this reason, we also use a different notation for the set of points contained in a cube, i.e.,

C e (z) = D ∞ e/2 (z) = y ∈ Q n δ : ∀j ∈ 1, . . . , n. |y j -z j | ≤ e 2
is the set of points contained in the cube that has edge length e ∈ Q δ (with e ≥ 0) and center z ∈ Q n δ .

Flat Cubes

A disadvantage of cubes is that they extend equally in all directions. This is helpful when we focus our search on all directions equally, e.g., when we are looking for an integer solution. If we are, however, focusing our search only on some directions, e.g., when we are looking for a mixed solution, then this fact becomes bothersome. For these problems, we would prefer objects that extend only in the directions that are relevant to our search. Our solution are cubes that expand only in the directions that correspond to integer variables and are flat in the directions that correspond to rational variables. We call these cubes flat cubes. As before, we formally define flat cubes by the set of points they contain, i.e., the flat cube with edge length e and center point z is defined as

F e (z) = y ∈ Q n 1 δ × {(z n 1 +1 , . . . , z n 1 +n 2 ) T } : dist d (z, y) ≤ e 2 .
Also as before, the flat cube F e (z) has a mixed solution if and only if it contains the closest n 2 -mixed point to its center: Lemma 2.9.3 (Flat Cube Mixed Solution). A flat cube F e (z) has a mixed solution if and only if it contains the closest n 2 -mixed point z n 2 to the center z.

Proof. The implication from left to right follows directly from Lemma 2.1.2. The implication from right to left is obvious.

We could also define flat d-norm balls. We refrain from this because we use d-norm balls only for comparison with cubes and the comparison between flat cubes and flat d-norm balls would not result in any additional information.

Basics of Transition Systems

(Unlabeled) transition systems-which are mathematically equivalent to abstract rewriting systems-are simply state based systems that change step by step via discrete transitions [START_REF] Klop | Term Rewriting Systems[END_REF]. 22 This means that transition systems can be used as an alternative to pseudocode for the formalization of algorithms. On the one hand, algorithms written as transition systems have the advantage of explicit definitions for their states and transitions. This makes them less error prone and it makes it easier to prove invariants over them. On the other hand, the description of an algorithm as pseudocode is much closer to an actual implementation of the respective algorithm. Naturally, this makes it easier to reimplement them and to estimate their behavior as part of an actual implementation. This is why we only present the most complex algorithms as transition systems, viz., CutSat, CutSat++, and CutSat g .

The formal description of a transition system must include the following information: (i) the set of states S over which the transition system operates; (ii) a subset of states S 0 ⊆ S called the start states of the transition system; (iii) a subset of states S e ⊆ S called the end states of the transition system; and (iv) a set of transitions called the transition relation R ⊆ (S \ S e ) × S of the transition system. This description also implicitly defines a directed graph, where the nodes are simply the states and the directed edges the transitions.

Typically, the set of states and transitions are infinite. This means we cannot explicitly list all of them but need to formally define them. In the case of transitions, we typically do so by defining transition rules. A definition of a transition rule looks abstractly as follows: Name of the Rule s =⇒ TS s if F (s, s ) In this definition, F (s, s ) are conditions that specify the valid input states s and output states s of the transitions defined by the rule. We also always label our transitions =⇒ TS with an abbreviation of the name of the transition system. In our example, this label is TS.

In the terms of an algorithm, start states define the valid input of our algorithm, end states the valid output of our algorithm, and the transitions, i.e., rule applications, are the operations our algorithm can take to modify the current state. The algorithms defined by transition systems are typically non-deterministic because states can have multiple outgoing transitions.

A run of our algorithm/transition system is a sequence of transitions, i.e., rule applications that is: (i) starting from one of the start states (the input of the run); and (ii) either diverging, i.e., has infinitely many rule applications, or terminating, i.e., ending in a state with no outgoing transitions (the output of the run). A transition system is terminating if there exists no run from any of the start states that is diverging. The correct output of a transition system is specified by a function f : S 0 × S e → {true, f alse}. A transition system is sound according to a specifying function if every terminating run for every start state s 0 ∈ S 0 ends in an end state s e ∈ S e such that f (s 0 , s e ) evaluates to true. The stuck states of a transition system are all terminating states that are not also end states. A transition system is complete if it is terminating, sound, and has no stuck states. Sometimes a transition system has too much non-determinism to uphold the invariants we want, e.g., termination and soundness. For this reason, we are allowed to define strategies for our transition system, i.e., some additional restrictions to our rule applications that are typically in the form of rule preferences. We then say that a transition system fulfills an invariant while following a given strategy if the invariant holds on all those runs that are still possible under the given strategy.

Chapter 3

CutSat++: A Complete and Terminating Approach to Linear Integer Solving

The SMT and theorem proving communities have developed several interesting approaches for their specific type of linear integer problems [START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF][START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF][START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF]. These approaches are based on a branch-and-bound strategy (see also Chapter 2.7.3), where the rational relaxation of an integer problem is used to cut off and branch on rational solutions1 . Together with the known a priori integer bounds for a problem, this branch-and-bound strategy yields a terminating and complete algorithm (see Chapter 2.8 and [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]). However, the a priori bounds grow exponentially in the number of inequalities. Even for toy examples the bounds easily exceed standard 64-Bit integer representations of today's computers. As a result, these bounds do not cause termination in a reasonable amount of time and checking the bounds is expensive as it requires big integer representations. Hence, the a priori bounds are often not integrated in implementations [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF]. Furthermore, the a priori bounds depend only on syntactic properties of a problem, such as the largest occurring constant and the number of inequalities. Therefore, it seems that a more promising termination approach needs to explore the inner structure of a problem.

The most well known calculus of this type is the CDCL (conflict-driven clause-learning) [START_REF] Bayardo | Using CSP look-back techniques to solve exceptionally hard SAT instances[END_REF][START_REF] Maaren | Handbook of Satisfiability[END_REF][START_REF] Moskewicz | Chaff: Engineering an efficient SAT solver[END_REF][START_REF] João | GRASP -a new search algorithm for satisfiability[END_REF] calculus for propositional satisfiability (SAT). It has changed SAT solving from a purely academic exercise to a standard verification technique in industry. The calculus starts by generating a partial model assumption by deciding (guessing) the truth value of propositional variables and by propagating new truth values with respect to the already set values and the problem clauses. If the model assumption evolves into an overall model, i.e., a model that satisfies all problem clauses, then satisfiability is shown. If it falsifies a clause, this clause together with the partial model assumption can be explored to derive a new learned clause via resolution. The learned clause is a logical consequence of the problem clauses, hence it explores the inner structure of the problem. The learned clause also repairs the failed partial model assumption with respect to the model generating algorithm. Furthermore, the learned clause is new by construction [START_REF] Weidenbach | Automated reasoning building blocks[END_REF] and constitutes progress on the basis of a wellfounded ordering. The latter two properties both independently guarantee termination of the CDCL calculus. For a toy example, consider the clauses P ∨ Q, ¬P ∨ ¬Q, ¬P ∨ Q. A run of the CDCL calculus may decide P to be true and then propagate ¬Q using the clause ¬P ∨ ¬Q. The idea of building explicit (partial) model assumptions guiding inferences is actually older than CDCL and goes back to the superposition calculus [START_REF] Bachmair | On restrictions of ordered paramodulation with simplification[END_REF]. It has meanwhile been transported to a variety of logics [START_REF] Alagi | NRCL -A model building approach to the Bernays-Schönfinkel fragment[END_REF][START_REF] Caferra | Automated Model Building[END_REF][START_REF] Piskac | Deciding effectively propositional logic using DPLL and substitution sets[END_REF] including arithmetic theories [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF][START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. From these model guided calculi, the CutSat calculus by Jovanović and de Moura [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] is the closest to our work and constitutes an important step towards a decision procedure that is both efficient and terminating on linear integer problems. Similar to CDCL and the other above mentioned calculi, termination relies on the generation of new inequalities that were derived based on a well-founded ordering. This means termination relies no longer on a priori bounds but on the exploration of the inner structure of a problem. If all variables of a problem are guarded-i.e., for every variable x i there are inequalities l i ≤ x i ≤ u i , where l i , u i are integer constants (see also Chapter 2.8)-then CutSat terminates. If there are unguarded variables, then CutSat may diverge or get stuck (see Section 3.3). This means, however, that the termination approach of the CutSat calculus is not working on all linear integer problems.

Our contribution in this chapter is an extension and refinement of the CutSat calculus, which we call CutSat++. In contrast to CutSat, Cut-Sat++ always terminates. The basic idea of both calculi is to reduce a problem containing unguarded integer variables to a problem containing only guarded variables. Unguarded variables are not eliminated. Instead, they are explored by learning further inequalities on smaller guarded variables to the problem. A strict total ordering on all variables, where all unguarded variables are larger than all guarded variables, provides the scale. After adding sufficiently many inequalities, feasibility of the problem depends only on guarded variables. Then a CDCL style calculus with explicit (partial) model building tests for feasibility by employing exhaustive propagation.

The most sophisticated part is to "turn" an unguarded variable into a guarded variable. The variable elimination procedure by Cooper [START_REF] David | Theorem proving in arithmetic without multiplication[END_REF] does so by replacing an unguarded variable with a case distinction that explores lower bounds and is represented by disjunctions. The lower bounds are derived from the problems inequalities. The procedure is provably terminating at the price of potentially exponentially growing coefficients and an exponentially growing Boolean problem structure. Still, the idea behind Cooper's procedure is our starting point and we will present a variation of the procedure, which is called weak Cooper elimination, in Section 3.4. Weak Cooper elimination does not create a complicated boolean structure because it considers not only the lower bounds but also the upper bounds derivable from the problems inequalities. By combining all pairs of inequalities that determine a lower and an upper bound for some variable, we get a new formula that determines whether the strictest lower bound is smaller than or equal to the strictest upper bounds for that variable without complicating the boolean structure. The only additional drawback is the introduction of new guarded variables and divisibility constraints. However, guarded variables seem to be less harmful, in practice, and divisibility constraints are needed for integer quantifier elimination anyway.

Satisfiability of linear integer problems is NP-complete. All algorithms known today need exponential time on certain classes of input problems. Since Cooper elimination and weak Cooper elimination do not care about the concrete structure of a given problem, the exponential behavior is almost guaranteed. The idea of CutSat++ is, therefore, to simulate a lazy variation of weak Cooper elimination. It is lazy because it does not apply a complete variable elimination step at once, but only partially whenever false inequalities (leading to so-called conflicting cores) occur over unguarded variables. Since CutSat++ uses a strategy that applies the rules according to a total variable order, which is closely related to the elimination order of a quantifier elimination procedure, it is not even necessary to actually remove the unguarded variables. Instead, the conflict is blocked by learning constraints in smaller variables according to the variable order. This leaves space for repairing model assumptions and applying simplification rules in order for the calculus to adapt to the specific structure of a problem and, hence, to systematically avoid certain cases of the worst case exponential behavior observed with Cooper elimination.

In more detail, CutSat++ starts with a partial model assumption for the linear integer problem. The model assumption is build by Decide (guess a value/bound) and Propagate (propagate values/bounds through inequalities to obtain new values/bounds) rules similar to CDCL, see Figure 3.1.

However, we are dealing with inequalities instead of propositional clauses. Variables no longer have just two possible assignments but exponentially many (even infinitely many if we ignore the existence of a priori bounds for the variables [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]). Therefore, the propagate rule assigns bounds x i ≤ b i instead of values x i = b i for some integer constant b i . This also means that a variable is only fixed, i.e., assigned to a specific value, if the partial model contains two bounds x i ≤ b i and x i ≥ b i . Now bound propagation takes an inequality and uses the bounds in the model assumption to propagate additional bounds. For example, the inequality 2x 1x 2 ≤ 0 together with the bound x 2 ≤ 1 propagates 2x 1 ≤ 1 or x 1 ≤ 1 2 . Since we are looking for an integer value for x 1 the actual bound becomes x 1 ≤ 0, which is also the bound CutSat++ would propagate. Another difference to SAT is that the number of propagation steps is no longer bound by the number of variables, but rather by the exponential a priori bounds for the variables. For example, let our model assumption contain the bounds x 1 ≥ 5 and x 2 ≥ 5 and let us look at the two inequalities 1-x 1 +x 2 ≤ 0 and x 1 -x 2 ≤ 0. We can use 1-x 1 +x 2 ≤ 0 ≡ 1+x 2 ≤ x 1 and x 2 ≥ 5 to propagate the bound x 1 ≥ 6. Then we use x 1x 2 ≤ 0 ≡ x 1 ≤ x 2 and x 1 ≥ 6 to propagate the bound x 2 ≥ 6. If we continue propagation using 1x 1 + x 2 ≤ 0 and x 1x 2 ≤ 0 alternately, we generate ever-growing bounds for x 1 and x 2 up to the exponential a priori bounds. Without adding the a priori bounds explicitly, bound propagation would even diverge in this example. This behavior is only possible because the problem contains unguarded variables. Although automatic detection of diverging bound propagations is possible in many cases, in particular in situations where subsequent propagation rule applications yield the divergence, it can also be the result of a combination of decisions and propagation rule applications. In this situation an automatic detection of the divergence gets far more complicated. To prevent ever-growing bounds, we will present a propagation strategy for CutSat++, see Definition 3.5.6, based on the already mentioned variable ordering that prevents divergence through propagation, even without a priori bounds.

As a result, CutSat++ generates after finitely many propagation and decision rule applications either a model for the problem, or some inequality is falsified by the partial model assumption. The analog situation in CDCL based SAT solving is the detection of a false clause. The CDCL calculus computes from the false clause via resolution steps the eventual learned clause that repairs the partial model assumption. The crucial invariant is that at any step the actual candidate clause remains false in the model assumption. The invariant guarantees that the model assumption will change eventually after backtracking parts of the model assumption and adding the learned clause. We can do something very similar for inequalities. The only difference is the resolution step, which is based on Fourier-Motzkin elimination instead of Boolean resolution. Let -a j x j + p j ≤ 0 and b j x j + q j ≤ 0 be two constraints over the integers in the focused representation (see Chapter 2.2.1), where a j , b j > 0 are constant integer coefficients and p j , q j are linear polynomials without x j . Then they-and every problem containing them-also imply the constraint b j • (-a j x j + p) + a j • (b j x j + q) ≤ 0 ≡ b j p + a j q ≤ 0 that does not contain the variable x j . For example, we can combine 1x 1 + x 2 ≤ 0 and x 1x 2 ≤ 0 through linear combination so the new constraint 1

• (1 -x 1 + x 2 ) + 1 • (x 1 -x 2 ) ≤ 0 ≡ 1 ≤ 0 directly
proves unsatisfiability of our problem. However, resolving conflicts is typically more challenging for linear integer constraints. For example, let our problem contain the inequalities -x 2 + 1 ≤ 0, -x 3 + 1 ≤ 0, 3x 1x 2 ≤ 0, and -3x 1 + x 3 ≤ 0 and let our current model include the bounds andx 3 ≤ 1 (decided) so x 2 and x 3 are both assigned to 1. We can use x 2 ≤ 1 and 3x 1x 2 ≤ 0 to propagate the bound x 1 ≤ 0 to our model. As a result, the inequality -3x 1 + x 3 ≤ 0 turns into a conflict because -3x 1 + x 3 ≤ 0 ≡ x 3 ≤ 3x 2 implies, together with the bounds in our current model, that 1 ≤ x 3 ≤ 3x 1 ≤ 0. However, resolving -3x 1 + x 3 ≤ 0 and 3x 1 -x 2 ≤ 0 with a linear combination results in the constraint -x 2 +x 3 ≤ 0 which is satisfied by our model. The resolution ended prematurely because the linear combination of the constraints -a j x j + p ≤ 0 and b j x j + q ≤ 0 is only guaranteed to be conflict preserving if one of the coefficients a j or b j is 1. Hence, we cannot resolve conflicts directly with the constraints used for propagation.

x 2 ≥ 1 (propagated from -x 2 + 1 ≤ 0), x 2 ≤ 1 (decided), x 3 ≥ 1 (propagated from -x 3 + 1 ≤ 0),
In order to overcome this problem, we will compute so-called tight justifications ±x j + r j ≤ 0 for the propagated bounds, where r j is a linear polynomial without x j . Tight justifications are inequalities entailed by our set of constraints that can propagate the same bounds but have coefficients ±1 for the propagated variables. For our example, the tight justifications are constructed as follows: The tight justifications for the bounds x 2 ≥ 1 and x 3 ≥ 1 are simply the propagating inequalities -x 2 + 1 ≤ 0 and -x 3 + 1 ≤ 0 because they already have -1 as the coefficient for the propagated variables. We generally get the tight justification for the bound x j ≤ b j propagated from a j x j + p j ≤ 0 by performing the following two steps: first we add to a j x j + p j ≤ 0 tight justifications from previously propagated bounds until the resulting inequality has the form a j x j + a j • r j + c j ≤ 0, where r j is a linear polynomial without x j and c j a constant integer; then we divide the combined inequality by the coefficient of the propagated variable x j and we get our tight justification x j +r j + c j a j ≤ 0. This means we get the tight justification for x j ≤ 0 by adding two times -x 2 + 1 ≤ 0 to 3x 1x 2 ≤ 0, which results in the inequality 3x 1 -3x 2 + 2 ≤ 0. Then we divide 3x 1 -3x 2 + 2 ≤ 0 by 3, which results in the inequality x 1x 2 + 1 ≤ 0. This inequality is now a tight justification for x 1 ≤ 0 because x 1x 2 + 1 ≤ 0 can propagate x 1 ≤ 0 from x 2 ≤ 1 and because its coefficient for x 1 is ±1. If we now resolve the conflicting inequality -3x 1 + x 3 ≤ 0 with the tight justification x 1x 2 + 1 ≤ 0 instead of 3x 1x 2 ≤ 0, then we get the inequality -3x 2 + x 3 + 3 ≤ 0. This inequality is still violated by our current model because 6

≤ x 3 + 3 ≤ 3x 2 ≤ 3.
Conflict resolution may also cause divergence in combination with unguarded variables. The reason is that any propagated bound can also be added to the model with a combination of decisions and conflict resolution. Therefore, we can repeat the divergent example from before even without the propagation rule. As a solution to this "conflict divergence", Cut-Sat++ introduces a second type of conflict resolution based on Cooper's quantifier elimination procedure. This new resolution is called unguarded conflict resolution. It uses the fact that we can transform any unguarded problem into a guarded one if we eliminate all unguarded variables with a quantifier elimination procedure. For efficiency reasons, however, the quantifier elimination rules are restricted to so-called conflicting cores which are condensed false inequalities generated out of the inequality falsified by the model assumption. CutSat++ enforces that all unguarded conflicts are handled by the new resolution, while all guarded conflicts are still handled solely by the conflict resolution motivated from CDCL and described above (see Section 3.2).

For the case of a conflicting core involving unguarded variables, we actually need to distinguish three subcases: (i) the conflicting core is the result of an inequality propagating a bound for an unguarded variable that then falsifies a second inequality, called an interval conflicting core, (ii) two inequalities have been propagating bounds for an unguarded variable falsifying a divisibility constraint, called a divisibility conflicting core, and (iii) a divisibility constraint is falsified although it still contains a variable that is unbounded in the model, called a diophantine conflicting core. For example, assume a divisibility constraint 6 | 4x j + r j where r j is a linear polynomial not containing x j and x j is unbounded. Now assume that the partial model assumption restricts the values of the variables in r j such that r j evaluates to 1. Then, the divisibility constraint becomes unsatisfiable although x j is still unbounded.

Altogether, (i) the CDCL style resolution of conflicts with purely guarded variables, (ii) the special resolution mechanisms for interval, divisibility and diophantine conflicting cores, and (iii) the exploration of an underlying well-founded ordering on the variables plus (iv) a strategy on the calculus rules yields the sound, complete and terminating calculus CutSat++. In Section 3.2, we start introducing CutSat++ formally as a transition system 2 by presenting the definitions and rules it shares with CutSat. These rules describe a CDCL style calculus and they constitute already a sound, complete, and terminating calculus for all problems that contain only guar-ded variables. In Section 3.3, we explain the rules and strategies used by CutSat to handle unguarded variables and show that they do not suffice. Our conclusion is that CutSat lacks, in addition to some refinements, a resolution case for diophantine conflicting cores. The basis for the exploration of this conflicting core is our new variant of Cooper elimination, called weak cooper elimination, which we present in Section 3.4. In this section, we also prove that any procedure that is based on weak Cooper elimination needs to consider diophantine conflicting cores for completeness (Theorem 3.4.7). In Sections 3.5-3.6, we then use weak Cooper elimination to define the inference rules of CutSat++ for the elimination of unguarded variables. The result is the sound, complete, and terminating CutSat++ calculus (see Section 3.6.3, Theorem 3.6.6 and Theorem 3.6.11). Finally, we give conclusions and point at possible directions for future research.

Related Work and Preliminaries

This chapter is based on a publication with Thomas Sturm and Christoph Weidenbach as co-authors [START_REF] Bromberger | Linear integer arithmetic revisited[END_REF]. An extended version of this publication has been accepted for publication in the Journal of Symbolic Computation.

The constraints in this chapter are either formatted according to the standard representation, i.e., a i1 x 1 + . . . + a in x n ≤ b i for inequalities and d i | a i1 x 1 +. . .+a in x n +b i for divisibility constraints (see also Chapter 2.2.1), or formatted according to the focused representation, i.e., a ij x j + p ij ≤ 0 for inequalities and d i | a ij x j + p ij for divisibility constraints (see also Chapter 2.2.1). Since this chapter handles a decision procedure for linear integer arithmetic, we also assume in this chapter that all variables are integer variables. For convenience, we abbreviate in this chapter integer satisfiability/equivalence/entailment with satisfiability/equivalence/entailment. Moreover, we assume that all coefficients a ij and constraint bounds b i are integer values. We are allowed to do so because of the equisatisfiability preserving transformations presented in Chapter 2.2.1.

This chapter builds on the basics of linear arithmetic (Chapter 2.2), on the concept of implied constraints (Chapter 2.5), on the definitions of (un)bounded and (un)guarded problems and variables (Chapter 2.8), and on the basics of transition systems (Chapter 2.10).

As already mentioned in the introduction to this chapter, our Cut-Sat++ calculus is an extension and refinement of the CutSat calculus by Jovanović and de Moura [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. The paper presenting CutSat is, therefore, the closest related work to this chapter. The similarities and differences between CutSat and CutSat++ can be summarized as follows: Both calculi can be split into two parts: (i) a CDCL style calculus, which we call CutSat g ; and (ii) an extension of the CDCL style calculus based on quantifier elimination techniques. The first part, CutSat g , was initially presented in [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] and is identical in both CutSat and CutSat++. 3 CutSat g is already on its own a terminating and complete calculus for all guarded problems; i.e., problems with only guarded variables. The second part (i.e., the quantifier elimination based extension) is necessary for handling problems containing unguarded variables. For this part, CutSat uses an extension called strong conflict resolution [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], which is not strict and exhaustive enough to guarantee termination on all unguarded problems (see Section 3.3). To resolve this, we developed the alternative extension unguarded conflict resolution for CutSat++ (Section 3.5) that does guarantee termination on all problems (including unguarded problems).

CutSat++ is also highly related to the IntSat calculus by Robert Nieuwenhuis [START_REF] Nieuwenhuis | The IntSat method for integer linear programming[END_REF]. IntSat is, similarly to CutSat g , a CDCL style calculus for linear integer arithmetic that only terminates on guarded problems. However, IntSat is in practice faster than CutSat g . The reasons are as follows: IntSat uses a weaker but faster version of conflict resolution (with respect to learning) that is still strong enough to be correct, complete, and efficient. Moreover, IntSat can add decided bounds that do not match any propagated bounds. Since IntSat and CutSat g are so similar, we should also be able to combine unguarded conflict resolution and IntSat to receive a complete and terminating calculus. We assume that the resulting combination would be more efficient in practice than CutSat++.

The Guarded Case

Our CutSat++ calculus can be separated into two parts: (i) CutSat g , a CDCL style calculus, which is already on its own a terminating and complete calculus for all guarded problems; and (ii) unguarded conflict resolution, an extension to CutSat g based on quantifier elimination techniques, which makes the calculus complete even if the problem contains unguarded variables. In this section, we will be presenting CutSat g in formal detail 4 .

This section is organized as follows: we define in Subsection 3.2.1 the states and models traversed by CutSat g and its extensions CutSat and CutSat++. In Subsection 3.2.2, we will explain the model construction via decisions and propagations. Besides the actual calculation of the propagated bound value via the function bound(J, x j , , M ), where ∈ {≤, ≥}, this also includes restrictions to our rules (e.g. improves(J, x j , , M )) that guarantee that our propagations always improve the model, i.e., our propagations always eliminate invalid solutions and never skip any valid solutions. We can, however, still skip valid solutions with decisions. That is why we present in Subsection 3.2.3 a CDCL-like conflict resolution to undo these decisions and find actual proofs of unsatisfiability. As mentioned in the introduction of this chapter, the Fourier-Motzkin based resolution for linear integer arithmetic does not work on arbitrary constraints, but only on tight justifications. 5 For this reason, we also introduce in Subsection 3.2.3 a secondary rule system (Fig. 3.3) that refines constraints used for propagation into tight justifications. In the CutSat g rules, this secondary system is called by the functions tight(J, x j , M ) and div-derive(D, x j , , M ), where ∈ {≤, ≥}. We conclude CutSat g with the Slack-Intro rule in Subsection 3.2.6, which prevents stuck states when encountering unguarded variables.

States and Models

CutSat++ decides satisfiability of a problem C. It either ends in the state unsat or in a state ν, sat , where ν is a satisfiable assignment for C. In order to reach one of those two end states, the calculus produces lower bounds x j ≥ b j and upper bounds x j ≤ b j for the variables in C. By L(x j , M ) = l j and U(x j , M ) = u j we denote the values l j , u j of the greatest lower bound x j ≥ l j and least upper bound x j ≤ u j for a variable x j in M , respectively. If there is no lower (upper) bound for x j in M , then L(x j , M ) = -∞ (U(x j , M ) = ∞). The definitions of U and L are extended to polynomials as done by [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]:

L(c, M ) = c; L(a • x j , M ) = a • L(x j , M ), if a > 0; L(a • x j , M ) = a • U(x j , M ), if a < 0; L(p + q, M ) = L(p, M ) + L(q, M ); U(c, M ) = c; U(a • x j , M ) = a • U(x j , M ), if a > 0; U(a • x j , M ) = a • L(x j , M ), if a < 0; U(p + q, M ) = U(p, M ) + U(q, M ).
The partial model M is complete if all variables x j are fixed in the sense that U(x j , M ) = L(x j , M ). In this case, we define ν[M ] : vars(C) → Z as the assignment that assigns to every variable x j the value L(x j , M ). By val(p, M ) := L(p, M ), we denote the value assigned to a fixed polynomial p. 6 This also means that val(p, M ) is defined only if all variables occurring in p are fixed in M .

We define our CutSat++ calculus in the form of a transition system that traverses states via rules. The rules are applied in a non-deterministic way. This type of presentation supports proofs by induction on the length of rule applications, where the rules can be considered independently. A state in CutSat++ is either one of the two types of end states ν, sat , unsat, or of the form S = M, C, I , where M is the current partial model, C is the current set of constraints, and I is either or an inequality entailed by C (C Z I) that constitutes a conflict, i.e., it evaluates to false under the partial model M . In case I is just , we will often abbreviate M, C, CutSat++ constructs all states M, C , I after the start state [[]], C in such a way that (i) C is equisatisfiable to C and that (ii) any satisfiable assignment ν for the current set of constraints C is also a satisfiable assignment for the original set of constraints C. Moreover, any partial model M generated by CutSat++ (i) stays consistent, i.e., L(x j , M ) ≤ U(x j , M ) for all variables x j ∈ vars(C), and (ii) improves, i.e., L(

x j , M ) > L(x j , M ) if M = [[M , x j ≥ b j ]] and U(x j , M ) < U(x j , M ) if M = [[M , x j ≤ b j ]].
Finally, no state generated by CutSat++ is stuck, i.e., a state generated by CutSat++ is either an end state or a rule is applicable. All of the above properties hold also for CutSat g and CutSat and we are going to elaborate why they hold step-by-step in the rest of this section.

Decisions and Propagations

We rely on decisions and propagations to construct a model (see Fig. 3.1). Via applications of the rule Decide, CutSat g adds decided bounds x j ≤ b j or x j ≥ b j , also called decisions, to the sequence M in state S [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. A decided bound assigns a variable x j to the lower or upper bound of x j in M . Via applications of the propagation rules, CutSat g adds propagated bounds x j ≥ I b j or x j ≤ I b j to the sequence M . To this end, the function bound(J, x j , , M ) defines the strictest lower bound (if = ≥) or upper bound (if = ≤) value b j that can be computed for constraint J under the partial model M , and the function tight(J, x j , M ) computes a corresponding justification I, i.e., a simplified version of the propagating constraint J that is needed for conflict resolution (see Subsection 3.2.3) and, therefore, annotated to the propagated bound x ≤ I b. 7 For an inequality J, bound(J, x j , , M ) is defined as follows:

Decide M, C =⇒ CS [[M, x j ≥ b j ]], C if U(x j , M ) = +∞, L(x j , M ) < b j = U(x j , M ) M, C =⇒ CS [[M, x j ≤ b j ]], C if L(x j , M ) = -∞, L(x j , M ) = b j < U(x j , M ) Propagate M, C =⇒ CS [[M, x j ≥ I b j ]], C if        J ∈ C is an inequality, improves(J, x j , ≥, M ), b j = bound(J, x j , ≥, M ), I = tight(J, x j , M ) M, C =⇒ CS [[M, x j ≤ I b j ]], C if        J ∈ C is an inequality, improves(J, x j , ≤, M ), b j = bound(J, x j , ≤, M ), I = tight(J, x j , M ) Propagate-Div M, C =⇒ CS [[M, x j ≥ I c j ]], C if            D = (d | a j x j + p j ) ∈ C, p j is fixed, improves(D, x j , ≥, M ), c j = bound(D, x j , ≥, M ), I = div-derive(D, x j , ≥, M ) M, C =⇒ CS [[M, x j ≤ I c j ]], C if            D = (d | a j x j + p j ) ∈ C, p j is fixed, improves(D, x j , ≤, M ), c j = bound(D, x j , ≤, M ), I = div-derive(D, x j , ≤, M ) Figure 3.1:
The decision and propagation rules of CutSat g bound(a j x j + p j ≤ 0,

x j , ≤, M ) =      - L(p j ,M ) a j if a j > 0, ∞ if a j ≤ 0. bound(a j x j + p j ≤ 0, x j , ≥, M ) =      -∞ if a j ≥ 0, - L(p j ,M ) a j if a j < 0.
An inequality J only propagates finite upper bounds for x j if coeff(J, x j ) > 0.

Symmetrically, an inequality J only propagates finite lower bounds for x j if coeff(J, x j ) < 0. For a divisibility constraint D, bound(D, x j , , M ) is defined as follows:

bound

(d | a j x j + p j , x j , , M ) =                        d a j L(x j ,M )+k j d -k j a j     if = ≥,     d a j U (x j ,M )+k j d -k j a j     if = ≤,
where a j > 0, d > 0, all variables in p j are fixed, and L(p j ) = k j . We defined the bound value bound(D, x j , , M ) for divisibility constraint D in such a way that CutSat g never skips a satisfiable solution for D:

Lemma 3.2.1 (No Skipping). Let D = d | a j x j + p j be a divisibility con- straint with a j > 0.
Let M, C be a state where the polynomial p j is fixed.

(1) Let b j = L(x j , M ) denote the current lower bound value for x j and let c j = bound(d | a j x j +p j , x j , ≥, M ) denote the lower bound value that we will propagate. Then bound(d | a j x j + p j , x j , ≥, M ) skips no satisfiable values for x j , i.e., d a j v j + L(p j , M ) holds for all v j such that b j ≤ v j < c j .

(2) Let b j = U(x j , M ) denote the current upper bound value for x j and let c j = bound(d | a j x j +p j , x j , ≤, M ) denote the upper bound value that we will propagate. Then bound(d | a j x j + p j , x j , ≤, M ) skips no satisfiable values for x j , i.e., d a j v j + L(p j , M ) holds for all v j such that c j < v j ≤ b j .

Proof. We only prove the case where c j = bound(d | a j x j + p j , x j , ≥, M ) denotes a lower bound. The proof for the second case is analogous. We assume for a contradiction that there exists a v j such that d | a j v j +L(p j , M ) and b j ≤ v j < c j are true. Since d | a j v j + L(p j , M ), it holds that

a j v j +k j d = a j v j +k j d and d a j v j +k j d -k j a j = d a j v j +k j d -k j a j = v j . Since L(x j , M ) = b j ≤ v j , it holds that a j L(x j ,M )+k j d ≤ a j v j +k j d . Hence, bound(d | a j x j + p j , x j , ≥, M ) = d a j b j +k j d -k j a j ≤ d a j v j +k j d -k j a j = v j .
Therefore, b j ≤ v j < c j is not true, which contradicts our initial assumption.

The rules of the CutSat g calculus are restricted in such a way that M stays consistent, i.e., L(x j , M ) ≤ U(x j , M ) for all variables x j ∈ vars(C). CutSat g also propagates only bounds that are more strict than the current bound for the variable x j , e.g., if CutSat g is going to propagate the lower bound x j ≥ b j , then the new lower bound b j must be strictly greater than the current lower bound L(x j , M ) for x j . This behaviour is expressed by the following predicate for constraints J: Although CutSat g constructs only consistent models M , it is possible that a constraint J is unsatisfiable under the model M . If constraint J is unsatisfiable under the model M in the state S = M, C, I , then we call J a conflict. An inequality p j ≤ 0 is a conflict if L(p j , M ) > 0; a divisibility constraint d | a j x j + p j is a conflict if all variables in p j are fixed and d a j b j + L(p j , M ) is unsatisfiable for all integers v j with L(x j , M ) ≤ v j ≤ U(x j , M ), i.e., there exists no integer value for x j within its current bounds such that d | a j x j + L(p j , M ) is satisfiable.

improves(J, x j , ≥, M ) = U(x j , M ) ≥ bound(J, x j , ≥, M ) > L(x j , M ) , improves(J, x j , ≤, M ) = L(x j , M ) ≤ bound(J, x j , ≤, M ) < U(x j , M ) .

(Guarded) Conflict Resolution

Conflict M, C =⇒ CS M, C, p j ≤ 0 if p j ≤ 0 ∈ C, L(p j , M ) > 0 Conflict-Div M, C =⇒ CS M, C, I if            J = (d | a j x j + p j ) ∈ C, p j is fixed, b j = U(x j , M ), bound(J, x j , ≥, M ) > b j , I = div-derive(J, x j , ≥, M ) M, C =⇒ CS M, C, I if            J = (d | a j x j + p j ) ∈ C, p j is fixed, b j = L(x j , M ), bound(J, x j , ≤, M ) < b j , I = div-derive(J, x j , ≤, M ) Skip-Decision [[M, γ]], C, I =⇒ CS M, C, I if    γ is a decided bound, I = (p j ≤ 0), L(p j , M ) > 0 Resolve [[M, γ]], C, I =⇒ CS M, C, I if    I = , γ is a propagated bound, I = resolve(γ, I) Backjump [[M, γ, M ]], C, J =⇒ CS [[M, γ ]], C if                γ is a decided bound, coeff(J, x j ) < 0, improves(J, x j , M ), I = tight(J, x j , M ), b j = bound(J, x j , ≥, M ), γ = x j ≥ I b j [[M, γ, M ]], C, J =⇒ CS [[M, γ ]], C if                γ is a decided bound, coeff(J, x j ) > 0, improves(J, x j , M ), I = tight(J, x j , M ), b j = bound(J, x j , ≤, M ), γ = x j ≤ I b j
Since CutSat g can use the Decide rules to introduce bounds that are not implied by the problem, encountering a conflict does not necessarily mean that the problem is unsatisfiable. It is also possible that the model became unsatisfiable because of a decided bound introduced by CutSat g . To find and undo those responsible decided bounds, CutSat g uses a CDCLlike conflict resolution, which we call guarded/standard conflict resolution (Fig. 3.2) [START_REF] Nieuwenhuis | Solving SAT and SAT modulo theories: From an abstract Davis-Putnam-Logemann-Loveland procedure to DPLL(T )[END_REF]. In CDCL, Boolean resolution is used to combine the current conflict C ∨ l with a clause used for unit propagation C ∨ l to receive a new conflict C ∨C without literals l or l. For CutSat g the function resolve:

resolve(x j I b j , J) = |a j |q j + |c j |p j ≤ 0 if a j • c j < 0, a j x + p j ≤ 0
otherwise , where I = c j x j + q j ≤ 0, J = a j x j + p j ≤ 0, and ∈ {≤, ≥}, fulfills a similar purpose: If the propagated bound x cx+q≤0 b is one of the bounds responsible for ax + p ≤ 0 being a conflict (i.e., a • c < 0), then resolve(x cx+q≤0 b, ax + p ≤ 0) combines the current conflict constraint ax + p ≤ 0 with the propagation justification cx + q ≤ 0 to the new conflict constraint |a|q + |c|p ≤ 0. If the propagated bound x cx+q≤0 b is not one of the responsible bounds (i.e., a • c ≥ 0), then resolve(x cx+q≤0 b, ax + p ≤ 0) ignores the propagation justification and simply returns/keeps the current conflict constraint ax+p ≤ 0. The function resolve is also the reason why we do not annotate propagated bounds with the actual constraints used for propagation, but with specially constructed justifications. To be more precise, the justifications are constructed in such a way that resolve is conflict preserving, i.e., J = resolve(γ, J) is a conflict in state M, C, J and C Z J if J = a j x j + p j ≤ 0 was a conflict in state [[M, γ]], C, J and C Z J [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. In CutSat g , the function resolve is conflict preserving because it requires that every justification I annotated to a bound γ = x j I b j is tight, i.e., justification I fulfills in state M, C, J the following conditions: Firstly, I is an inequality and

C Z I. Secondly, if = ≤, then coeff(I, x j ) = 1; if = ≥, then coeff(I, x j ) = -1. Finally, bound(I, x j , , M )
b, i.e., the justification I implies at least a bound as strong as x j I b j .

CutSat g calculates the justification I for the bound x j I b j ( ∈ {≤, ≥}) propagated from inequality J = ±a j x j + p j ≤ 0 with the function tight(J, x j , M ) = I defined by the set of rules in Figure 3.3. A state in this rule system is a pair M , ±a j x j + a j s ⊕ r , where a j > 0, s and r are polynomials, and M is a prefix of the initial M , i.e., M = [[M , M ]]. The start state for tight(±a j x j + p j ≤ 0, x j , M ) is M, ±a j x j ⊕ p j and the corresponding end state is the tight linear justification for the inequality ±a j x j + p j ≤ 0. The first goal of the tight Consume M, ±a j x j + a j s ⊕ a j a i x i + r =⇒ tight M, ±a j x j + a j s j + a j a i x i ⊕ r , where

x j = x i . Resolve-Implied [[M, γ]], ±a j x j + a j s ⊕ p =⇒ tight M, ±a j x j + a j s j ⊕ q ,
where γ is a propagated bound and q ≤ 0 = resolve(γ, p ≤ 0).

Decide-Lower

[[M, x i ≥ b i ]], ±a j x j + a j s ⊕ a i x i + r =⇒ tight M, ±a j x j + a j s + a j c i x i ⊕ r + (a j c i -a i )q , where x i ≤ I b i in M , with I = x i + q ≤ 0, and c i = a i a j . Decide-Lower-Neg [[M, x i ≥ b i ]], ±a j x j + a j s ⊕ a i x i + r =⇒ tight M, ±a j x j + a j s ⊕ a i q + r , where x i ≤ I b i in M , with I = x i -q ≤ 0, and a i < 0. Decide-Upper [[M, x i ≤ b i ]], ±a j x j + a j s ⊕ a i x i + r =⇒ tight M, ±a j x j + a j s + a j c i x i ⊕ r + (a i -a j c i )q , where x i ≥ I b i in M , with I = -x i + q ≤ 0, and c i = a i a j . Decide-Upper-Pos [[M, x i ≤ b i ]], ±a j x j + a j s ⊕ a i x i + r =⇒ tight M, ±a j x j + a j s ⊕ a i q + r
, where x i ≥ I b i in M , with I = -x i + q ≤ 0, and a i > 0. Round (and terminate) M, ±a j x j + a j s ⊕ b j =⇒ tight ± x j + s + b j a j ≤ 0 Figure 3.3: Rule system that derives tightly propagating inequalities [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] rule system is to produce an inequality where all coefficients are divisible by a j = coeff(J, x j ). To this end, we apply rules (Figure 3.3) that linearly combine the constraint we want to tighten with tightened constraints from previous propagations. The process stops as soon as the polynomial on the right side of ⊕ becomes empty. In this case, all coefficients are divisible by a j = coeff(J, x j ) and we derive the justification I with the final rule Round. The tight-transition-system is guaranteed to derive a tight inequality because there exist appropriate bounds with tight justifications for all variables occurring in inequality J (except x j ) or CutSat g could not propagate a bound for J.

CutSat g calculates the justification I for the bound x j I b j ( ∈ {≤, ≥}) propagated from divisibility constraint D = d | a j x j + p j with the function div-derive(D, x j , , M ). To do so, CutSat g uses the fact that any bound propagated for D can also be propagated with an intermediate step by the two auxiliary inequalities -dz + a j x j + p j ≤ 0 and dza j x jp j ≤ 0, which are implied by D. For instance, let us look at the lower bound value:

bound(D, x j , ≥, M ) =     d a j b j +k j d j -k j a j    
,

where D = d | a j x j + p j , a j > 0, d > 0, p j is fixed, k j = L(p j , M
), and b j = L(x j , M ). First we split the diophantine representation8 dz = a j x j +p j of the divisibility constraint D into two inequalities: -dz + a j x j + p j ≤ 0 and dza j x jp j ≤ 0. Then we see that the subterm c =

a j b j +k j d
is equal to the bound value computable from one of the split inequalities: bound(-dz + a j x j + p j ≤ 0, z, ≥, M ) =

a j b j +k j d
. The fitting justifications for the two types of subterms are abbreviated with div-part(D,

x j , M ): div-part(D, x j , ≥, M ) = tight(-dz + a j x j + p j ≤ 0, z, M )
is the justification for the lower bound subterm c =

a j b j +k j d and div-part(D, x j , ≤, M ) = tight( dz -a j x j -p j ≤ 0, z, M )
is the justification for the upper bound subterm c = a j b j +k j d

. For div-part, we forbid tight to apply the Consume rule to the variables x j and z. This restriction to the Consume rule guarantees that the resulting inequality ±z+ r ≤ 0 = div-part(D, x j , M ) does not contain the variable x j . Given I 2 = -z + r ≤ 0 = div-part(D, x j , M ) and I 1 = dza j x jp j ≤ 0, we use resolve(x j ≥ I 2 c, I 1 ) = -a j x j + drp j ≤ 0 = I 3 to receive the inequality that computes the complete lower bound:

bound(I 3 , x j , ≥, M ) = d L(r,M )-k j a j ≥     d a j b j +k j d j -k j a j    
.

Finally, we compute the justification for the actual divisibility constraint

D = d | a j x j + p j with the function div-derive(D, x j , M ), which is defined as div-derive(D, x j , ≥, M ) = tight(-a j x j + dr -p j ≤ 0, x j , M ), where -z + r ≤ 0 = div-part(D, x j , ≥, M ), and div-derive(D, x j , ≤, M ) = tight( a j x j + dr + p j ≤ 0, x j , M ), where z + r ≤ 0 = div-part(D, x j , ≤, M ). Learn M, C, I =⇒ CS M, C ∪ {I}, I if I ∈ C ∪ { } Forget M, C ∪ {J} =⇒ CS M, C if C Z J,

Learning

The original set of constraints C implies all constraints I derived during the standard conflict resolution, i.e., all constraints I in a conflict state M, C, I visited by CutSat g . Therefore, we can add any such constraint I to our constraint set C and still get an equivalent set of constraints C = C ∪ {I}.

CutSat g can do this too with the rule Learn (Fig. 3.4). Extending our constraint set with learned constraints is not necessary for termination or completeness. However, learning constraints is useful in practice because the learned constraints might propagate bounds that our original constraints cannot propagate. Still, we do not want to add all conflict constraints I from every conflict state M, C, I to our constraint set as this would also slow down CutSat g . In practice, we recommend to only learn those conflicts J from which CutSat g applies the Backjump rule. Moreover, we recommend to use the rule Forget (Fig. 3.4) to remove in regular intervals those learned constraints that have not propagated enough bounds.

Reaching the End States

CutSat g ends a run when it has determined whether the original constraint set has an integer solution or not. Formally, CutSat g does so with the rules Sat, Unsat-Div, and Unsat (Fig. 3.5). Sat is called as soon as CutSat g has found a complete model M , i.e., a model where all variables are fixed, that also satisfies all constraints in our current constraint set C. The assignment ν[M ] in the end state υ[M ], sat is then a satisfiable assignment for the current and the original constraint set. The rules Unsat-Div and Unsat are applicable as soon as CutSat g derives a trivially unsatisfiable constraint, i.e., either a constant inequality b j ≤ 0 that is unsatisfiable or a divisibility constraint d | a 1 x 1 + . . . + a n x n + c that can never be true because gcd(d, a 1 , . . . , a n ) c. The unsatisfiability of the original constraint set is then marked by the end state unsat.

If we apply the rules according to the following strategy, then CutSat g is sound, complete, and terminates for guarded problems, i.e., those input problems C containing only guarded variables: Definition 3.2.2 (Reasonable Strategy [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]). A strategy is reasonable if Propagate applied to constraints of the form ±x jb j ≤ 0 has the highest priority over all rules and the Forget Rule is applied only finitely often.

Sat M, C =⇒ CS υ[M ], sat if ν[M ] satisfies C Unsat-Div M, C =⇒ CS unsat if d | a 1 x 1 + . . . + a n x n + c ∈ C, gcd(d, a 1 , . . . , a n ) c Unsat M, C, b j ≤ 0 =⇒ CS unsat if b j > 0 Figure 3.5:
The rules of CutSat g that lead to end states If the problem is unguarded, i.e., the input problem C contains at least one unguarded variable, then CutSat g with a reasonable strategy can diverge. In the next Section, we will discuss why CutSat g diverges when there are unguarded variables and why its extension CutSat cannot prevent all divergent behavior. But before move on to CutSat, we extend CutSat g by one final rule.

As mentioned before, CutSat g might diverge on unguarded problems. However, CutSat g might also get stuck on unguarded problems, i.e., reach a state that is not an end state but where no rule is applicable. For this reason, we are going to add the rule Slack-Intro (Fig. 3.6) to CutSat g .

Until we add Slack-Intro to CutSat g , stuck states are possible because CutSat g is sometimes unable to propagate and, thereby, also not decide any bounds for a variable x j . In this case, we call the variable x j stuck. More formally, a variable x j is called stuck in state S = M, C if M contains no bounds for x j and we can neither use Propagate or Propagate-Div to add a bound for x j . Guarded variables x j are never stuck because they have by definition constraints of the form ±x jb j ≤ 0 ∈ C, which CutSat g is always able to propagate. The easiest example where stuck variables result in a stuck state is the state [[]], {x 1x 2 ≤ 0} . Here both variables are stuck, so we can never construct a model.

In case all unfixed variables x j are stuck, Slack-Intro allows us to add new constraints to the problem that make one of the variables unstuck. To this end, Slack-Intro exploits the following fact: an input problem C is only satisfiable if it is satisfiable inside an a priori fixed finite interval for all variables, i.e., if there exists an x S ≥ 0 such that C ∪ {-x S ≤ x i ≤ x S } is also satisfiable. Slack-Intro simulates this fact in CutSat g by adding a new variable x S and some constraints {-x S ≤ 0, x ix S ≤ 0, -x ix S ≤ 0} to C such that the bounds -x S ≤ x i ≤ x S can be propagated after x S is propagated and decided. This also means that CutSat g expands the bounds for all previously stuck variables incrementally and symmetrically from the integer constant zero towards ±∞. Slack-Intro alone actually prevents all stuck states for CutSat g .

Slack-Intro

M, C =⇒ CS M, C if               
x j is stuck, all other unfixed variables in M, C are also stuck, x S is the slack-variable,

C = C ∪ {-x S ≤ 0, x j -x S ≤ 0, -x j -x S ≤ 0} Figure 3.6:
The Slack-Intro rule of CutSat g

Divergence of CutSat

An easy example for divergence-already stated by [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]-relies on cyclic dependencies during propagation:

Example 3.3.1. Let S i = M i , C (i ∈ N
) be a series of states defined by:

C := {-x 1 ≤ 0 Ix 1 , -x 2 ≤ 0 Ix 2 , -x 3 ≤ 0 Ix 3 , 1 -x 1 + x 2 ≤ 0 J 1 , x 1 -x 2 -x 3 ≤ 0 J 2 } M 0 := [[x 1 ≥ Ix 1 0, x 2 ≥ Ix 2 0, x 3 ≥ Ix 3 0, x 3 ≤ 0]] M i+1 := [[M i , x 1 ≥ J 1 i + 1, x 2 ≥ J 2 i + 1]]
CutSat g with a reasonable strategy has diverging runs starting in state S 0 = [[]], C . Let CutSat g traverse the states S 0 , S 0 , S 1 , S 2 , . . . in the following fashion: CutSat g reaches state S 0 := M 0 , C from state S 0 after propagating the constraints I x 1 , I x 2 , I x 3 , and fixing x 3 with a decided bound. CutSat g reaches state S i+1 from state S i after:

• applying Propagate to J 1 to propagate γ

x 1 i+1 := x 1 ≥ J 1 i + 1 so that M i := [[M i , γ x 1 i+1 ]] and S i := M i , C • applying Propagate to J 2 to propagate γ x 2 i+1 := x 2 ≥ J 2 i + 1 so that M i+1 := [[M i , γ x 2
i+1 ]] and S i+1 := M i+1 , C To summarize, this example shows how we can exploit the cyclic dependence between x 1 and x 2 to increase the lower bounds for x 1 and x 2 to an arbitrarily large value i ∈ N.

To prevent this type of divergence, Jovanović and de Moura suggest to restrict the rules Propagate and Propagate-Div so they only propagate those bounds that are δ-relevant. Definition 3.3.2 (δ-relevant [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]). Let nb(x j , M ) be the number of bounds for x j in M . Let δ > 0 and nb max ∈ N be two parameters that are fixed at the start of the CutSat g run. Then the new lower bound

x j ≥ J b j is δ-relevant in the state M, C if: • L(x j , M ) = -∞, or • U(x j , M ) = ∞, or • L(x j , M ) + δ| L(x j , M )| < b j and nb(x j , M ) < nb max . Likewise, the new upper bound x j ≤ J b j is δ-relevant in the state M, C if: • U(x j , M ) = ∞, or • L(x j , M ) = -∞, or • U(x j , M ) -δ| U(x j , M )| > b j and nb(x j , M ) < nb max .
If we now restrict the rules Propagate and Propagate-Div as suggested, then δ-relevance gives us the following guarantees: The first case of δ-relevance guarantees that CutSat g can still propagate at least one bound for every variable. The second case of δ-relevance guarantees that a bounded variable (with x ju j ≤ 0 ∈ C and -x j + l j ≤ 0 ∈ C) can be propagated without restrictions because its guards l j ≤ x j ≤ u j already guarantee that there are at most |u jl j + 1| propagations for x j . The last case of δ-relevance guarantees that an unbounded variable is at most propagated nb max times and, thus, prevents propagation divergence. The additional conditions L(x j , M ) + δ| L(x j , M )| < b j and U(x j , M ) -δ| U(x j , M )| > b j prevent CutSat g from wasting its limited amount of propagations on those that change the bounds only slightly. We also do not introduce any stuck states with our restriction to δ-relevance because any bound x j ≥ J b j that the third case of δ-relevance prevents from being propagated can also be introduced to the bound sequence M if we first fix x j with Decide, then apply Conflict(-Div) to J, and finally add the bound x j ≥ J b j to M with Backjump.

However, this simulation of Propagate(-Div) via Decide and conflict resolution also means that divergence is not eliminated but only shifted divergence from propagation to conflict resolution. We can also use Example 3.3.1 for this type of divergence; but this time we simulate propagation as described above via conflict resolution. Instead of applying Propagate between the states S i and S i+1 , we fix one variable at a time with a decided bound. If we fix x 1 to L(x 1 , M i ) with a decided bound, we directly detect a conflict in J 1 . We enter conflict resolution by applying Conflict to J 1 , which can be exited directly with the Backjump rule. This removes the decided bound on x 1 and adds instead x 1 ≥ J 1 i + 1 on top of the bound sequence. Hence, we reach the intermediate state S i without applying the rule Propagate. Analogously, we do the same for x 2 with J 2 and reach the state S i+1 without applying the rule Propagate.

The best way to handle this type of divergence is to forbid standard conflict resolution-i.e., the rules Conflict, Conflict-Div, Resolve, Backjump, Skip-Decision, Unsat, and Learn-from handling constraints containing unguarded variables. As an alternative to normal conflict resolution, Jovanović and de Moura suggested a secondary conflict analysis called strong conflict resolution [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. Strong conflict resolution is also the predecessor of the unguarded conflict resolution we are going to present in Section 3.5. Strong conflict resolution is based on the fact that we can transform any unguarded problem into a guarded one if we eliminate all unguarded variables with a quantifier elimination procedure. Instead of eliminating all unguarded va- The strong conflict resolution rules by [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] riables before we apply CutSat g , strong conflict resolution introduces two new rules (see Figure 3.7) that extend the CutSat g calculus by a second type of conflict resolution, which applies quantifier elimination in-between the applications of the original CutSat g rules. We call CutSat g with this extension CutSat. The newly added rules do not apply the complete quantifier elimination algorithm, nor do they apply it at all possible instances, but only on minimal conflicts containing unguarded variables. These minimal conflicts are also called conflicting cores. [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]). Let S = M, C , C ⊆ C, let x j be a variable in C , let all other variables in C be fixed, let a j , b j > 0, and let l j = bound(-a j x j + p j ≤ 0, x j , ≥, M ) and u j = bound(b j x jq j ≤ 0, x j , ≤ , M ). The pair (x, C ) is a conflicting core if it is of one of the following two forms:

Resolve-Cooper M, C =⇒ CS M , C ∪ R if            (x j , C ) is a conflicting core, x j is unguarded, x j is the minimal conflicting variable, (R y , R c ) = cooper(x j , C ), R = R y ∪ R c Solve-Div M, C =⇒ CS M, C if    divisibility constraints I 1 , I 2 ∈ C, {I 1 , I 2 } = div-solve(x j , {I 1 , I 2 }), C = C \ {I 1 , I 2 } ∪ {I 1 , I 2 } div-solve(x j , {d 1 | a 1j x j + p 1j , d 2 | a 2j x j + p 2j }) = {d 1 d 2 | dx j + c 1j d 2j p 1j + c 2j d 1j p 2j , d | -a 1j p 2j + a 2j p 1j },

Definition 3.3.3 (Conflicting Cores

(1) C = {-a j x j + p j ≤ 0, b j x jq j ≤ 0} and l j > u j , i.e., the lower bound from -a j x j + p j ≤ 0 contradicts the upper bound from b j x jq j ≤ 0; in this case, (x j , C ) is called an interval conflicting core and its strong resolvent is

({-y i ≤ 0, y i -a j + 1 ≤ 0}, {b j p j -a j q j + b j y i ≤ 0, a j | y i + p j }) (2) C = {-a j x j + p j ≤ 0, b j x j -q j ≤ 0, d | c j x j + s j }
and l j ≤ u j , and for all v j ∈ [l j , u j ] ∩ Z we have d c j v j + L(s, M ), i.e., there exists no value for x j within the bounds defined by the two inequalities such that the divisibility constraint becomes satisfiable; in this case, (x j , C ) is called a divisibility conflicting core and its strong resolvent is ({-

y i ≤ 0, y i -m ≤ 0}, {b j p j -a j q j + b j y i ≤ 0, a j | y i + p j , a j d | c j p j + a j s j + c j y i })
In both cases, y i is a fresh variable and m = lcm a j ,

a j d gcd(a j d,c j ) -1.
We refer to the respective strong resolvents for a conflicting core (x j , C ) by the function cooper(x j , C ), which returns a strong resolvent (R y , R c ) as defined above, Definition 3.3.3. These strong resolvents (R y , R c ) are simply the constraints that we get after we eliminate ∃x j ∈ Z from ∃x j ∈ Z. C with Cooper's quantifier elimination procedure [START_REF] David | Theorem proving in arithmetic without multiplication[END_REF]. Note that the newly introduced variable y i is guarded by the constraints in R y . In Section 3.5, we will extend the definition of conflicting cores by a third type of core called a diophantine conflicting core. In the final example at the end of this section, we will also show why this core is necessary to guarantee termination.

Besides conflicting cores and strong resolvents, strong conflict resolution also relies on a total order ≺ over all variables such that x i ≺ x j for all guarded variables x i and unguarded variables x j . In relation to Cooper's quantifier elimination, the order ≺ describes the elimination order for the unguarded variables, viz., x i ≺ x j if x j is eliminated before x i . A variable x j is called maximal in a constraint I if x j is contained in I, i.e., coeff(I, x j ) = 0, and all other variables x i in I are smaller, i.e., x i ≺ x j . The maximal variable in I is also called its top variable (x j = top(I)). If there is a conflicting core (x j , C ) in some state S, then x j is called a conflicting variable. The conflicting variable is minimal in the state S if there exists no conflicting variable x i in S such that x i ≺ x j .

Instead of actually eliminating a conflicting variable or the conflicting core (x j , C ), the rule Resolve-Cooper (see Figure 3.7) only adds the constraints R y ∪ R c from the strong resolvent (R y ,R c ) to the problem. By doing so, Resolve-Cooper is supposed to guarantee that the old conflicting core (x j , C ) is always ignored in favor of a "smaller" conflicting core. The following strategy guarantees that strong conflict resolution and the guarded conflict resolution interact as little as possible. To be more precise, guarded conflict resolution is supposed to be applied to guarded conflicts, i.e., conflict constraints containing only guarded variables, and strong conflict resolution is supposed to be applied to unguarded conflicts, i.e., conflict constraints containing unguarded variables. As a consequence, the strategy is also supposed to guarantee termination for the whole calculus: Definition 3.3.4 (Two-layered Strategy). We say a strategy is two-layered if

• it is reasonable (Definition 3.2.
:= { -x 1 ≤ 0 I 1 , -x 2 ≤ 0 I 2 , -x 3 ≤ 0 I 3 , x 3 ≤ 0 I 4 , x 3 + 1 ≤ 0 I 5 , 1 -x 1 + x 2 ≤ 0 J 1 , x 1 -x 2 ≤ 0 J 2 } be a problem. Let S i = M i , C for i ∈ N be a series of states with: M 0 := [[x 1 ≥ I 1 0, x 2 ≥ I 2 0, x 3 ≥ I 3 0, x 3 ≤ I 4 0]], M i+1 := [[M i , x 1 ≥ J 1 i + 1, x 2 ≥ J 2 i + 1]].
Let the variable order be given by x 3 ≺ x 2 ≺ x 1 . CutSat with a two-layered strategy has diverging runs starting in state S 0 = [[]], C . Let CutSat traverse the states S 0 , S 0 , S 1 , S 2 , . . . in the following fashion: CutSat reaches state S 0 from state S 0 after propagating the constraints I 1 , I 2 , I 3 , and I 4 . CutSat reaches state S i+1 from state S i after:

• fixing x 1 to i with the decided bound γ

x 1 d := x 1 ≤ i so that M 1 i := [[M i , γ x 1 d ]] and S 1 i := M 1 i , C • applying Conflict to the constraint J 1 because L(1 -x 1 + x 2 , M 1 i ) > 0 so that M 2 i := M 1 i and S 2 i := M 2 i , C, J 1 • undoing the decided bound γ x 1
d by applying Backjump; this results in the exchange of γ x 1 d with the bound γ

x 1 = x 1 ≥ J 1 i + 1 so that M 3 i := [[M i , γ x 1 ]] and S 3 i := M 3 i , C • fixing x 2 to i with the decided bound γ x 2 d := x 2 ≤ i so that M 4 i := [[M 3 i , γ x 2 d ]] and S 4 i := M 4 i , C • applying Conflict to the constraint J 2 because L(x 1 -x 2 , M 4 i ) > 0 so that M 5 i := M 4 i and S 5 i := M 5 i , C, J 1 • undoing the decided bound γ x 2
d by applying Backjump; this results in the exchange of γ x 2 d with the bound γ

x 2 = x 2 ≥ J 2 i + 1 so that M 6 i := [[M 3 i , γ x 2 ]] and S i+1 = S 6 i := M 6 i , C Notice that (x 3 , {I 3 , I 5 }) is a conflicting core in the states S i , S 1
i , and S 4 i , and, therefore, the variable x 3 is the minimal conflicting variable in those states. Since I 3 and I 4 bound x 3 , the conflicting core is also guarded. Therefore, Resolve-Cooper as defined by [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] is not applicable, which in turn implies that Conflict is applicable. This means the two-layered strategy was not strict enough to prevent standard conflict resolution from being applied to unguarded conflicts.

A straightforward fix to Example 3.3.5 is to limit the application of the Conflict and Conflict-Div rules to guarded constraints. Our second example shows that CutSat can still diverge by infinitely many applications of the Solve-Div rule.

k+1 := d k 2 for k ∈ N, let C 0 = {4 | 2x 1 +2x 2 , 2 | x 1 +x 3 } be a problem, and let S 0 = [[]],
C 0 be the start CutSat state. Let the variable order be given by x 1 ≺ x 2 ≺ x 3 . Then CutSat has divergent runs S 0 =⇒ CS S 1 =⇒ CS S 2 =⇒ CS . . .. For instance, let CutSat apply the Solve-Div rule whenever applicable. By an inductive argument, Solve-Div is applicable in every state S n = [[]], C n , and the constraint set C n has the following form:

C n = {2d n | d n x 1 + d n x 2 , d n | dn 2 x 2 -dn 2 x 3 } if n is odd, {2d n | d n x 1 + d n x 2 , d n | dn 2 x 1 + dn 2 x 3 } if n is even.
Therefore, CutSat applies Solve-Div infinitely often and diverges. If we were to simplify the constraints in C n , then we would even see that the constraints C n for an even n are always just

{4 | 2x 1 + 2x 2 , 2 | x 1 + x 3 } and the constraints C n for an odd n are always just {4 | 2x 1 + 2x 2 , 2 | x 2 -x 3 }.
This means that we are cycling between the same two sets of constraints.

A straightforward fix to Example 3.3.6 is to limit the application of Solve-Div to maximal variables in the variable order ≺. Our third example shows that CutSat can apply Conflict and Conflict-Div infinitely often. Example 3.3.7 differs from Example 3.3.5 in that the conflicting core contains also unguarded variables.

Example 3.3.7. Let C := { -x 1 ≤ 0 I 1 , -x 2 ≤ 0 I 2 , -x 3 ≤ 0 I 3 , x 3 ≤ 0 I 4 , 1 -x 1 + x 2 + x 3 ≤ 0 J 1 , x 1 -x 2 -x 3 ≤ 0 J 2 } be a problem. Let S i = M i , C
for i ∈ N be a series of states with:

M 0 := [[x 1 ≥ I 1 0, x 2 ≥ I 2 0, x 3 ≥ I 3 0, x 3 ≤ I 4 0]], M i+1 := [[M i , x 1 ≥ J 1 i + 1, x 2 ≥ J 2 i + 1]].
Let the variable order be given by x 3 ≺ x 1 ≺ x 2 . CutSat with a two-layered strategy has diverging runs starting in state S 0 = [[]], C . For instance, let CutSat traverse the states S 0 , S 0 , S 1 , S 2 , . . . in the following fashion: CutSat reaches state S 0 from state S 0 after propagating the constraints I 1 , I 2 , I 3 and I 4 . CutSat reaches state S i+1 from state S i after:

• fixing x 1 to i and x 2 to i with decided bounds γ

x 1 d := x 1 ≤ i and γ x 2 d := x 2 ≤ i so that M 1 i := [[M i , γ x 1 d , γ x 2 d ]] and S 1 i := M 1 i , C • applying Conflict to J 1 because L(1 -x 1 + x 2 + x 3 , M 1 i ) > 0 so that M 2 i := M 1 i and S 2 i := M 2 i , C, J 1 • undoing the decided bounds γ x 2
d and γ x 1 d by applying Skip-Decision and then Backjump; the result is the sequence

M 3 i := [[M i , γ x 1 ]] and the state S 3 i := M 3 i , C , where γ x 1 = x 1 ≥ J 1 i + 1 • fixing x 2 to i and x 1 to i + 1 with decided bounds γ x 2 d := x 2 ≤ i and γ x 1 d := x 2 ≤ i + 1 so that M 4 i := [[M 3 i , γ x 2 d , γ x 1 d ]] and S 4 i := M 4 i , C • applying Conflict to J 2 because L(x 1 -x 2 -x 3 , M 4 i ) > 0 so that M 5 i := M 4 i and S 5 i := M 5 i , C, J 1
• undoing the decided bounds γ x 1 d and γ x 2 d by applying Skip-Decision and then Backjump; the result is the sequence

M 6 i := [[M 3 i , γ x 2 ]
] and the state S i+1 = S 6 i := M 6 i , C , where γ x 2 = x 2 ≥ J 2 i + 1. Whenever we fix x 1 and x 2 with Decide to their current respective lower bounds, we turn (x 3 , {J 1 , J 2 }) into a conflicting core. As a result, x 3 is the minimal conflicting variable in the states S 1 i and S 4 i , which in turn prevents the application of Resolve-Cooper and admits the application of Conflict. Therefore, the two-layered strategy was again not strict enough to prevent standard conflict resolution from being applied to unguarded conflicts.

Applying the fix suggested for Example 3.3.5 to Example 3.3.7 results in a stuck state. Here, a straightforward fix is to change the definition of conflicting cores to cover only those cores where the conflicting variable is the maximal variable. 9We could also prevent Examples 3.3.5 and 3.3.7 if we were to fix the decision order to the variable order. However, for the following counterexample, fixing the decision order is not enough:

Example 3.3.8. Let C := { -x 3 ≤ 0 I 1 , x 3 ≤ 0 I 2 , -x 4 ≤ 0 I 3 , x 4 ≤ 0 I 4 , -x 5 ≤ 0 I 5 , -x 5 + x 1 + x 3 ≤ 0 I 6 , -x 5 -x 1 -x 3 ≤ 0 I 7 , -x 5 + x 2 + x 4 ≤ 0 I 8 , -x 5 -x 2 -x 4 ≤ 0 I 9 , 2 + 3x 1 -4x 2 + 3x 3 -4x 4 ≤ 0 J 1 , -1 + 3x 1 -4x 2 + 3x 3 -4x 4 ≤ 0 J 2 , -1 -3x 1 + 2x 2 -3x 3 + 2x 4 ≤ 0 J 3
} be a problem. Let the variable order be given by

x 3 ≺ x 4 ≺ x 5 ≺ x 1 ≺ x 2 .
CutSat has a divergent run starting in state S 0 = [[]], C if it follows the following strategy: Let x j be the smallest unfixed variable.

• Propagate (if possible) the strictest upper bound for x j , • if J 1 is a conflict, then fix all unfixed variables with the rule Decide to their current upper bound, and start the standard conflict analysis with the rule Conflict, • otherwise propagate the strictest lower bound for x j and fix x j with a decision to the current lower bound Notice that we prevent the application of Resolve-Cooper by first fixing all variables and then shadowing x 1 and x 2 with the guarded variables x 3 and x 4 , respectively. If we change the definition of conflicting cores, as proposed for Example 3.3.7, then the divergent behaviour described in Example 3.3.8 is not possible. Thus, fixing the decision order to the variable order is no real alternative to the fixes proposed before.

The fixes that we suggested for the above examples are restrictions to CutSat which have the consequence that Conflict(-Div) cannot be applied to unguarded constraints, Solve-Div is applicable only for the elimination of the maximal variable, and the conflicting variable x 1 is the maximal variable in the associated conflicting core C . However, our next and final example shows that these restrictions are too strong and therefore lead to stuck states.

Example 3.3.9. Let CutSat include restrictions to maximal variables in the definition of conflicting cores and in the Solve-Div rule as described above. Let there be additional restrictions in CutSat to the rules Conflict and Conflict-Div such that these rules are only applicable to conflicts that contain no unguarded variable. Let

C := {-x 1 ≤ 0 I 1 , x -1 ≤ 0 I 2 , -x 2 ≤ 0 I 3 , 6 | 4x 2 + x 1 J } be a problem. Let M := [[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0, x 1 ≥ 1, x 2 ≤ 0]]

Weak Cooper Elimination

In order to fix the stuck state of Example 3.3.9 in the previous section, we are going to introduce in Section 3.5 a new conflicting core, which we call diophantine conflicting core. For understanding diophantine conflicting cores, as well as further modifications to be made, it is helpful to understand the connection between CutSat++ and a variant of Cooper's quantifier elimination procedure [START_REF] David | Theorem proving in arithmetic without multiplication[END_REF].

The original Cooper elimination takes a variable x j , a problem C, and produces a disjunction of problems equivalent to ∃x j ∈ Z.C:

∃x j ∈ Z.C ≡ 0≤y i <m C -∞ {x j → y i } ∨ -a j x j +p j ≤0 ∈ C 0≤y i <a j •m {a j | p j + y i } ∪ C{x j → p j +y i a j } , where a > 0, m = lcm{d ∈ Z : (d | a j x j + p j ) ∈ C}, C -∞ = ⊥ if
there exists a constraint of the form -a j x j + p j ≤ 0 ∈ C, and, otherwise,

C -∞ = {(d | a j x j + p j ) ∈ C}.
Although Cooper elimination gets rid of the variable x j , the elimination also comes at a price. One application of Cooper elimination results in a disjunction of quadratically many problems out of a single problem. Moreover, we have to somehow "remove" the fractions p j +y i a j used to replace x j because division is not part of the linear arithmetic language. We achieve this by multiplying each constraint with a j . However, this "removal" of the fractions causes a worst-case quadratic increase in the absolute size of the coefficients. The number of disjunctions and the size of coefficients have even a worst-case exponential increase if we look at several iterations of Cooper elimination. Now that we have formally defined Cooper elimination and stated its structural properties, we will explain in an intuitive way why Cooper elimination actually works. To this end, let us look at the result of applying Cooper elimination to a small example:

Example 3.4.1. Let C := { 2 | x 1 + x 2 J 1 , 2 | x 1 + x 3 J 2 , -3x 1 + x 2 ≤ 0 J 3 , -2x 1 + 2x 2 -2 ≤ 0 J 4 , x 1 -4x 3 ≤ 0 J 5 }
be the initial problem. Then the result of applying Cooper elimination is:

[∃x 1 ∈ Z.C ] ≡ 0≤y 1 <1 C -∞ ∨ 0≤y 1 <6 {3 | y 1 + x 2 } ∪ C J 3 ∨ 0≤y 1 <4 {2 | y 1 + 2x 2 -2} ∪ C J 4 ,
where

C -∞ := ⊥ C J 3 := C{x 1 → x 2 +y 1 3 } := { 6 | y 1 + 4x 2 , 6 | y 1 + x 2 + 3x 3 , -y 1 ≤ 0, -2y 1 + 4x 2 -6 ≤ 0, y 1 + x 2 -12x 3 ≤ 0} , C J 4 := C{x 1 → 2x 2 -2+y 1 2 } := { 4 | y 1 + 4x 2 -2, 4 | y 1 + 2x 2 + 2x 3 -2, -3y 1 -4x 2 + 6 ≤ 0, -y 1 ≤ 0, y 1 + 2x 2 -8x 3 -2 ≤ 0} .
A linear arithmetic problem has an integer solution if and only if the strictest lower bound for x j is an integer solution for the problem. In Cooper elimination, we use this fact and do a case distinction over the strictest lower bound for x j via a disjunction. First, we assume that there exists no lower bound for x j . We express this case with the subformula C -∞ . Since both J 3 and J 4 define a lower bound for

x 1 (x 1 ≥ x 2 3 and x 1 ≥ 2x 2 -2 2
, respectively), C -∞ simplifies to ⊥. Next, we assume that J 3 is the basis for the strictest lower bound x 1 ≥ l 1 , i.e., l 1 := l 1 + y 1 (with y 1 ≥ 0) is the smallest integer larger than or equal to l 1 := x 2 3 such that 2 | l 1 + x 2 and 2 | l 1 + x 3 are true. We also know that l 1 + 2 is a strict upper bound for l 1 because 2

| l 1 + 2y 1 + x 2 ≡ 2 | l 1 + x 2 , 2 | l 1 + 2y 1 + 1 + x 2 ≡ 2 | l 1 + 1 + x 2 , 2 | l 1 + 2y 1 + x 3 ≡ 2 | l 1 + x 3 , 2 | l 1 + 2y 1 + 1 + x 3 ≡ 2 | l 1 + 1 + x 3 .
Therefore, l 1 is either l 1 or l 1 + 1. Since the ceiling function is not part of our syntax, we cannot assign x 1 directly to x 2 3 or x 2 3 + 1. However, we know that x 2 3 is one of the three values

x 2 3 , x 2 +1 3 , or x 2 +2
3 and that x 2 3 + 1 is one of the three values

x 2 +3 3 , x 2 +4 3 , or x 2 +5
3 . Hence, we do another case distinction (via a disjunction and the factor y 1 ) and replace x 1 with x 2 +y 1 3 . The result is the subformula 0≤y 1 <6 {3 | y 1 + x 2 } ∪ C J 3 . Finally, we assume that J 4 is the basis of the strictest lower bound x 1 ≥ 2x 2 -2

3

. Again, we do a case distinction (via a disjunction and the factor y 1 ) and replace x 1 with

2x 2 +y 1 -2 2
. The result is the subformula

0≤y 1 <4 {2 | y 1 + 2x 2 -2} ∪ C J 4 .
Our notion of weak Cooper elimination is a variant of Cooper elimination, which is very helpful for understanding problems around CutSat. The idea is, instead of building a disjunction over all potential solutions for x j , to add additional guarded variables and constraints without x j that guarantee the existence of a solution for x j . We assume here that C contains exactly one divisibility constraint for x j . If there exists no divisibility constraint for x j , then we can simply add the trivially true divisibility constraint 1 | x j . If there exist multiple divisibility constraint for x j , then we use the following function to transform every set of constraints C into an equivalent set of constraints C that contains only one divisibility constraint for

x j : div-solve(x j , {d 1 | a 1j x j + p 1j , d 2 | a 2j x j + p 2j }) = {d 1 d 2 | dx j + c 1j d 2j p 1j + c 2j d 1j p 2j , d | -a 1j p 2j + a 2j p 1j }, where d = gcd(a 1j d 2 , a 2j d 1 )
, and c 1j and c 2j are integer values such that c 1j a 1j d 2 + c 2j a 2j d 1 = d [START_REF] David | Theorem proving in arithmetic without multiplication[END_REF][START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. The function div-solve(x j , {D 1 , D 2 }) takes a set of two divisibility constraints {D 1 , D 2 } containing x j and returns an equivalent set of two divisibility constraints {D 1 , D 2 }, where only one constraint contains x j [START_REF] David | Theorem proving in arithmetic without multiplication[END_REF][START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. Therefore, we can replace every pair of divisibility constraints {D 1 , D 2 } containing x j with the pair returned from div-solve(x j , {D 1 , D 2 }) to decrease the number of divisibility constraints containing x j by one. It follows that exhaustive application of div-solve to all divisibility constraints containing x j removes all such constraints except one. Now weak Cooper elimination takes a variable x j , a problem C with only one divisibility constraint d | c j x j + s j ∈ C for x j , and produces a new and equivalent problem by replacing ∃x j ∈ Z.C with:

∃Y. {I ∈ C : coeff(I, x j ) = 0} ∪ {gcd(c j , d) | s j } ∪ y i ∈Y R y i
where y i ∈ Y is a newly introduced variable for every pair of constraints -a j x j + p j ≤ 0 ∈ C and b j x jq j ≤ 0 ∈ C with a j , b j > 0,

R y i = { -y i ≤ 0, y i -m ≤ 0, b j p j -a j q j + b j y i ≤ 0, a j | y i + p j , a j d | c j p j + a j s j + c j y i }
is a resolvent for the same inequalities, where m := lcm a j , a j d gcd(a j d,c j ) -1, and ∃Y abbreviates the sequence of quantified variables ∃y 1 , . . . , y m ∈ Z contained in Y = {y 1 , . . . , y m }. The major difference between Cooper elimination and weak Cooper elimination is that one introduces disjunctions and the other variables. For our purposes variables are more beneficial because each new variable y i is guarded by the constraints in R y i . Hence, we can eliminate unguarded variables without changing the conjunctive structure of our problems.

Weak Cooper elimination works informally because the transformation determines whether there exists an assignment ν for all variables except x j such that the strictest lower and upper bounds for x j under ν still contain an integer solution. For a more detailed explanation, let ν be a satisfiable assignment for the formula after one weak Cooper elimination step on C. Then we compute a strictest lower bound x j ≥ l j and a strictest upper bound x j ≤ u j from C for the variable x j under the assignment ν. We now argue that there is a value for x j such that x j ≥ l j , x j ≤ u j , and d | c j x j + s j are all satisfied. Whenever l j = -∞ and u j = ∞, the bounds x j ≥ l j , x j ≤ u j are given by constraints of the form -a j x j + p j ≤ 0 ∈ C and b j x jq j ≤ 0 ∈ C, respectively, so that l j = ν(p j ) a j and u j = ν(q j ) b j .

In this case, the extension of ν with ν(x j ) = ν(y i +p j ) a j satisfies C because the constraint a j | y i + p j ∈ R y i guarantees that ν(x j ) ∈ Z, the constraint b j p ja j q j + b j y i ≤ 0 ∈ R y i guarantees that l j ≤ ν(x j ) ≤ u j , and the constraint a j d | c j p j + a j s j + c j y i ∈ R y i guarantees that ν also satisfies d | c j x j + s j ∈ C. In all other cases, i.e., when l j = -∞ or u j = ∞, we extend ν by an arbitrary small or alternatively large value for x j that satisfies d | c j x j + s j ∈ C. There exist arbitrarily small (large) solutions for x j and d | c j x j + ν(s j ) because gcd(c j , d) | s j is satisfied by ν.

Now let us look at the result of applying Cooper elimination to the problem

C from Example 3.4.1: Example 3.4.2. Let C := { 2 | x 1 + x 2 J 1 , 2 | x 1 + x 3 J 2 , -3x 1 + x 2 ≤ 0 J 3 , -2x 1 + 2x 2 -2 ≤ 0 J 4 , x 1 -4x 3 ≤ 0 J 5 }
be the initial problem. We first have to use div-solve to simplify C because C contains two divisibility constraints (J 1 and J 2 ) for x 1 . Applying div-solve(x 1 , {J 1 , J 2 }) returns the two divisibility constraints 4 | 2x 1 + 2x 2 and 2 | x 2x 3 , which we will use to replace J 1 and J 2 in C . The result is the new, but equivalent problem

C := { 4 | 2x 1 + 2x 2 J 1 , 2 | x 2 -x 3 J 2 , -3x 1 + x 2 ≤ 0 J 3 , -2x 1 + 2x 2 -2 ≤ 0 J 4 , x 1 -4x 3 ≤ 0 J 5 },
which contains only one divisibility constraint (J 1 ) for x 1 . Now we will split the application of weak Cooper elimination into three steps. First, we select all constraints I ∈ C without the variable

x 1 : {I ∈ C : coeff(I, x 1 ) = 0} := {2 | x 2 -x 3 }. Next, we take the divisibility constraint 4 | 2x 1 + 2x 2 and eliminate x 1 from it : (∃x 1 ∈ Z. 4 | 2x 1 + 2x 2 ) ≡ (gcd(2, 4) | 2x 2 ) ≡ (2 | 2x 2 )
. Finally, we construct resolvents for every pair -a j x j + p j ≤ 0 ∈ C and b j x jq j ≤ 0 ∈ C: R y

1 := {-y 1 ≤ 0, y 1 -5 ≤ 0, y 1 + x 2 -12x 3 ≤ 0, 3 | y 1 + x 2 , 12 | 2y 1 + 8x 2 }
for J 3 and J 5 , and

R y 2 := { -y 2 ≤ 0, y 2 -3 ≤ 0, y 2 + 2x 2 -8x 3 -2 ≤ 0, 2 | y 2 + 2x 2 -2, 8 | 2y 2 + 8x 2 -4}
for J 4 and J 5 . The combination of these constraints is then the result of applying weak Cooper elimination:

[∃x 1 ∈ Z.C ] ≡ [∃x 1 ∈ Z.C] ≡ [∃y 1 ∈ Z.∃y 2 ∈ Z. ({J 2 } ∪ {2 | 2x 2 } ∪ R y 1 ∪ R y 2 )] .
CutSat++ performs weak Cooper elimination not in one step but subsequently adds to the states the constraints from the resolvents R y i as well as the divisibility constraint gcd(c j , d) | s j with respect to a strict ordering on the unguarded variables. The extra divisibility constraint gcd(c j , d) | s j in weak Cooper elimination is necessary whenever the problem C has no constraint of the form -a j x j + p j ≤ 0 ∈ C or b j x jq j ≤ 0 ∈ C. For example:

Example 3.4.3. Let C = {x 2 -1 ≤ 0, -x 2 + 1 ≤ 0, 6 | 2x 1 + x 2 }
be a problem and x 1 be the unguarded variable we want to eliminate. As there are no inequalities containing x 1 , weak Cooper elimination without the extra divisibility constraint returns C = {x 2 -1 ≤ 0, -x 2 + 1 ≤ 0}. While C has a satisfiable assignment ν(x 2 ) = 1, C has not since 2x 1 + 1 is never divisible by 6.

The following equivalence states the correctness of weak Cooper elimination:

∃x j ∈ Z.C ≡ ∃Y. {I ∈ C : coeff(I, x j ) = 0} ∪ {gcd(c j , d) | s j } ∪ y i ∈Y R y i
For any R y i introduced by weak Cooper elimination, we can also show the following Lemma:

Lemma 3.4.4 (Divisibility Core Resolvent). Let y i be a new variable. Let a j , b j , c j > 0. Then (∃x j ∈ Z.{-a j x j + p j ≤ 0, b j x j -q j ≤ 0, d | c j x j + s j }) ≡ (∃y i ∈ Z.{-y i ≤ 0, y i -m ≤ 0, b j p j -a j q j + b j y i ≤ 0, a j | y i + p j , a j d | c j p j + a j s j + c j y i }).
Proof. See [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] pp. 101-102 Lemma 4.

That means satisfiability of the respective R y i guarantees a solution for the triple of constraints it is derived from. An analogous Lemma holds for the divisibility constraint gcd(c j , d) | s j introduced by weak Cooper elimination:

Lemma 3.4.5 (Diophantine Core Resolvent). Let c j > 0. Then (∃x j ∈ Z.d | c j x j + s j ) ≡ gcd(c j , d) | s j .
Proof. We equivalently rewrite the divisibility constraints into diophantine equations, viz., d | c j x j + s j and gcd(c j , d) | s j into ∃z i ∈ Z.dz ic j x j = s j and ∃y i ∈ Z. gcd(c j , d)y i = s j , respectively. We choose d ∈ Z and c j ∈ Z such that d • gcd(c j , d) = d and c j • gcd(c j , d) = c j , respectively. Next, we differentiate between the two directions of the equivalence. Firstly, let ν be a variable assignment such that dν(z i )c j ν(x j ) = ν(s j ) and, therefore, also

d | c j ν(x j ) + ν(s j ). Hence, ν(s j ) = dν(z i ) -c j ν(x j ) = gcd(c j , d) • (d j ν(z i ) -c j ν(x j )). Therefore, extending ν with ν(y i ) = (d ν(z i ) -c j ν(x j
)) leads to an assignment ν that also satisfies gcd(c j , d)y i = s j .

Secondly, let ν be a variable assignment such that gcd(c j , d)ν(y i ) = ν(s j ) holds and, therefore, also gcd(c j , d) | ν(s j ). By Bézout's Lemma [START_REF] Bézout | Théorie générale des équations algébriques[END_REF], there exist a j , b j ∈ Z such that a j db j c j = gcd(c j , d). Hence,

a j dν(y i ) -b j c j ν(y i ) = (a j d -b j c j )ν(y i ) = gcd(c j , d)ν(y i ) = ν(s j )
. Therefore, extending ν with ν(z i ) = a j ν(y i ) and ν(x j ) = b j ν(y i ) leads to an assignment ν that also satisfies dz ic j x j = s j .

Hence, satisfiability of gcd(c j , d) | s j guarantees a solution for the divisibility constraint d | c j x j + s j . The rule Resolve-Weak-Cooper (Figure 3.9) in our CutSat++ calculus exploits these properties by lazily generating the resolvents R y i and the constraint gcd(c j , d) | s j in the form of unguarded resolvents. Furthermore, it is not necessary for the divisibility constraints to be a priori reduced to one, as done for weak Cooper elimination. Instead, the rules Solve-Div-Left and Solve-Div-Right (Figure 3.9) perform lazy reduction. Now let us return to the completeness proof of weak Cooper elimination. In this proof, we will use an assignment ν for all variables but x j . Since such an assignment ν fixes all variables but x j , we can determine the satisfiability of a problem C by looking at the extensions of ν for x j . To this end, we will define the solution set S ⊆ Z for variable x j , problem C, and assignment ν as the set of values v k ∈ S that extends ν to a satisfiable solution for C, i.e., C{x j → v k } (v k ∈ S) is satisfied by ν. If we look, for instance, at the solution set S d of a divisibility constraint, then we find the following useful property: Lemma 3.4.6 (Divisibility Solution Sets). Let ν be an assignment for all variables except x j . Let S d be the solution set for variable x j , assignment ν, and constraint d | c j x j + s j . Then

S d = ∅ or S d = {v 0 + kv : k ∈ Z} for some v 0 ∈ Z, v ∈ Z \ {0}
. This means that S d is either empty or unbounded from above and below.

Proof. If S d = ∅, then there exists a value v 0 ∈ S d such that d | c j v 0 + ν(s j ). We first prove that there exists a v ∈ Z\{0} such that d | c j (v 0 +kv )+ν(s j ) for all k ∈ Z and, therefore, v 0 + kv ∈ S d . We choose v and k such that v := d gcd(c j ,d) and k := c j gcd(c j ,d) . Then we deduce for any k ∈ Z:

d | c j (v 0 + kv ) + ν(s j ) ≡ d | c j v 0 + ν(s j ) + c j kv ≡ d | c j v 0 + ν(s j ) + c j k d gcd(c j ,d) ≡ d | c j v 0 + ν(s j ) + dkk ≡ d | c j v 0 + ν(s j ) It remains to show that for every v k ∈ S d there exists a k ∈ Z such that v 0 + kv = v k . As S d is the solution set, we know that both d | c j v 0 + ν(s j ) and d | c j v k + ν(s j ) are true. Therefore, d | c j v 0 + ν(s j ) -(c j v k + ν(s j )) ≡ d | c j (v 0 -v k ). As d = v gcd(c j , d), the term c j (v 0 -v k ) is only divisible by d if v 0 -v k is divisible by v . Therefore, ∃k ∈ Z.v 0 -v k = kv .
We need Lemma 3.4.6 in the correctness proof of weak Cooper elimination to show that we can choose an arbitrary small or large solution for x j that satisfies d | c j x j + ν(s j ). As mentioned in the outline of the proof, the ability to choose arbitrary small and large solutions for x j is necessary when C contains no constraints of the form -a j x j + p j ≤ 0 or b j x jq j ≤ 0. Theorem 3.4.7 (Weak Coooper Correctness). Let C be a problem with exactly one divisibility constraint I d with a non-zero coefficient coeff(I d , x j ) for x j . Then

∃x j ∈ Z.C ≡ ∃Y. {I ∈ C : coeff(I, x j ) = 0} ∪ {gcd(c j , d) | s j } ∪ y i ∈Y R y i
where y i ∈ Y is a newly introduced variable for every pair of constraints -a j x j + p j ≤ 0 ∈ C and b j x jq j ≤ 0 ∈ C with a j , b j > 0,

R y i = { -y i ≤ 0, y i -m ≤ 0, b j p j -a j q j + b j y i ≤ 0, a j | y i + p j , a j d | c j p j + a j s j + c j y i }
is a resolvent for the same inequalities, where m := lcm a j , a j d gcd(a j d,c j ) -1, and ∃Y abbreviates the sequence of quantified variables ∃y 1 , . . . , y m ∈ Z contained in Y = {y 1 , . . . , y m }.

Proof. First, we partition the problem C as follows:

C l = {-a j x j + p j ≤ 0 ∈ C : a j > 0}, C u = {b j x j -q j ≤ 0 ∈ C : b j > 0}, I d = d | c j x j + s j ∈ C, C r = {I ∈ C : coeff(I, x j ) = 0}. By Lemma 3.4.4, it holds for all -a j x j + p j ≤ 0, b j x j -q j ≤ 0 ∈ C with a j , b j > 0 that: (∃x j ∈ Z.C) → (∃x j ∈ Z.{-a j x j + p j ≤ 0, b j x j -q j ≤ 0, d | c j x j + s j }) → (∃y i ∈ Z.R y i ), where
R y i = { -y i ≤ 0, y im ≤ 0, b j p ja j q j + b j y i ≤ 0, a j | y i + p j , a j d | c j p j + a j s j + c j y i }. Because of Lemma 3.4.5, it also holds that:

(∃x j ∈ Z.C) → (∃x j ∈ Z.d | c j x j + s j ) → gcd(c j , d) | s j . Since C r ⊆ C, it
also holds that: (∃x j ∈ Z.C) → C r . As all new variables y i ∈ Y appear only in one resolvent R y i , the above implications prove

∃y i ∈ Z.C → ∃Y. {I ∈ C : coeff(I, x j ) = 0} ∪ {gcd(c j , d) | s j } ∪ y i ∈Y R y i .
Assume, vice versa, that ν is a satisfiable assignment for the formula after one step of weak Cooper elimination. Then it is easy to deduce the following facts:

• Let S l be the solution set for x j , ν, and

I l = -a j x j + p j ≤ 0 ∈ C with a j > 0. Then S l = ν(p j ) a j , ν(p j ) a j + 1, . . . . • Let S u
be the solution set for x j , ν, and

I u = b j x j -q j ≤ 0 ∈ C with b j > 0. Then S u = . . . , ν(q j ) b j -1, ν(q j ) b j .
• Let S I be the solution set for x j , ν, and C l ∪C u . Then S I = I l ∈C l S l ∩ Iu∈Cu S u .

• Let the set S I be bounded from below, i.e., S I = {l, l + 1, . . .} or S I = {l, . . . , u}. Then l = max I∈C l ν(p j ) a j : I = -a j x j + p j ≤ 0 .

• Let the set S I be bounded from above, i.e., S I = {. . . , u -1, u} or S I = {l, . . . , u}. Then u = min I∈Cu ν(q j ) b j : I = b j x jq j ≤ 0 .

• By Lemma 3.4.5, d | c j x j + ν(s j ) is satisfiable because gcd(c j , d) | s j is contained in the result formula of weak Cooper elimination. By Lemma 3.4.6, the set of solutions for x j , ν, and d | c j x j + s j has the form S d = {v 0 + kv : k ∈ Z}. • The solution set S for x j , ν, and C is S = S d ∩ S I . Next, we do a case distinction on the structure of C:

• Let C l = ∅, then S I is unbounded from below. We choose a small enough v k ∈ S d , i.e., small enough k ∈ Z such that v k = v 0 + kv . Then the assignment x j → v k and x i → ν(x i ) (if x i = x j ) satisfies C . • Let C u = ∅, then S I is unbounded from above. We choose a large enough v k ∈ S d , i.e., large enough k ∈ Z such that v k = v 0 + kv . Then the assignment x j → v k and x i → ν(x i ) for all x i = x j satisfies C . • Let |C l |, |C u | > 0. We select I l = -a j x j + p j ≤ 0 such that ν(p j ) a j = max I∈C l ν(p j ) a j : I = -a j x j + p j ≤ 0 and I u = b j x j -q j ≤ 0 such that ν(q j ) b j = min I∈Cu ν(q j ) b j : I = b j x j -q j ≤ 0 .
The resolvent for the two constraints I l and I u is

R y i = { -y i ≤ 0, y i -m ≤ 0, b j p j -a j q j + b j y i ≤ 0, a j | y I + p j , a j d | c j p j + a j s j + c j y i }.
We will now extend ν to also include ν(x j ) = ν(p j +y i ) a j and show that ν(p j +y i ) a j is in the set of solutions S of C. All of the remaining deductions stem from the evaluation of the resolvent under ν. Since a j | ν(p j + y i ),

ν(p j +y i ) a j ∈ Z. Moreover, since ν(p j +y i ) a j ∈ Z and ν(b j p j -a j q j + b j y i ) ≤ 0, ν(p j +y i ) a j ∈ S I = ν(p j ) a j , . . . , ν(q j ) b j . Fi- nally, since a j d | ν(c j p j + a j s j + c j y i ) = a j d | a j c j ν(x j ) + a j ν(s j ) = d | c j ν(x j ) + ν(s j ), it follows that ν(p j +y i ) a j ∈ S d . Hence, ν satisfies C .
We stated that weak Cooper elimination can only be applied to those problems where C contains one divisibility constraint d | a j x j + p j in x j . To expand weak Cooper elimination to any set of constraints C , we briefly explained how to exhaustively apply div-solve to eliminate all but one constraint d | a j x j + p j for x j . The algorithm CombDivs(x j , C ) (Figure 3.8) is a more detailed version of this procedure.

| c j x j + s j ∈ C such that c j = 0 1 C d := {d | c j x j + s j ∈ C : c j > 0} 2 C := C \ C d ; 3 if (C d = ∅) then 4 return C ∪ {1 | x j } ; 5 while (|C d | > 1) do 6 Select d 1 | a 1j x j + p 1j , d 2 | a 2j x j + p 2j ∈ C d ; 7 C d := C d \ {d 1 | a 1j x j + p 1j , d 2 | a 2j x j + p 2j }; 8 d = gcd(a 1j d 2j , a 2j d 1j ) ; 9 Choose c 1j and c 2j such that c 1j a 1j d 2j + c 2j a 2j d 1j = d; 10 C d := C d ∪ {d 1 d 2 | dx j + c 1j d 2 p 1j + c 2j d 1 p 2j } ; 11 C := C ∪ {d | -a 1j p 2j + a 2j p 1j }; 12 end 13 return C ∪ C d ;
Figure 3.8: An algorithm that combines all divisibility constraints containing x j until only one such constraint remains Proof. Follows directly from the proof of equivalence of the div-solve transformation [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF].

The relationship between CutSat++ and weak Cooper elimination is as follows: On the left side of the equivalence in Theorem 3.4.7 there occur two combinations of constraints: triples consisting of two inequations with a divisibility constraint and single divisibility constraints. Each triple of constraints {-a j x j + p j ≤ 0, b j x jq j ≤ 0, d | c j x j + s j } ∈ C is a potential divisibility conflicting core 10 , i.e., a set of constraints that can turn into a conflicting core if p j , q j , s j are fixed and the constraints are contradictory for variable x j . Moreover, the single divisibility constraint {d | c j x j + s j } ∈ C is a potential diophantine conflicting core, i.e., a divisibility constraint that can turn into a conflicting core if it becomes contradictory for variable x j after s j is fixed. On the right side of the equivalence in Theorem 3.4.7, the resolvents R y i for the respective triple of constraints and the resolvent gcd(c j , d) | s j for the divisibility constraint are equivalent to the unguarded resolvent that CutSat++ introduces for the appropriate conflicting cores. However, compared to CutSat++, weak Cooper elimination introduces all resolvents at once and, thereby, ensures that the resolvents subsume all potential conflicting cores. Therefore, we can replace all constraints containing x j with the resolvents for conflicting cores over x j and receive an equisatisfiable formula (see Theorem 3.4.7). This means that Theorem 3.4.7 shows that no other conflicting cores are necessary besides interval, divisibility, and diophantine conflicting cores. Moreover, Example 3.4.3 shows that we need to consider diophantine conflicting cores-and not just interval and divisibility conflicting cores-to preserve equisatisfiability.

The preprocessing step described in the CombDivs(x j , C ) algorithm can also be found in CutSat and CutSat++. However, CutSat and Cut-Sat++ perform CombDivs(x j , C ) lazily; the former with the rule Solve-Div, the latter with the rules Solve-Div-Left and Solve-Div-Right.

Unguarded Conflict Resolution

Weak Cooper elimination is capable of creating an equisatisfiable problem where satisfiability depends only on guarded variables. We simulate it in a lazy manner by extending CutSat g with three new rules, which we call the unguarded conflict resolution rules (Figure 3.9). The now extended rule system is our calculus CutSat++. Instead of eliminating all unguarded variables before the application of CutSat++, the rules perform the same intermediate steps as weak Cooper elimination, viz., the combination of divisibility constraints via div-solve and the construction of resolvents, to resolve and block conflicts in unguarded constraints. As a result, CutSat++ can avoid some of the intermediate steps of weak Cooper elimination. Moreover, CutSat++ is not required to apply the intermediate steps of weak Cooper elimination one variable at a time and does not eliminate unguarded variables. This has the advantage that CutSat++ might find a satisfiable assignment or detect unsatisfiability without encountering and resolving a large number of unguarded conflicts. As a result, the number of divisibility constraint combinations and introduced resolvents might be much smaller in the lazy approach of CutSat++ than during the elimination with weak Cooper elimination. Only in the worst case, CutSat++ has to perform all of weak Cooper elimination's intermediate steps. Then the strictly-twolayered strategy (Definition 3.5.7) guarantees that CutSat++ recognizes that all unguarded conflicts have been produced.

In order to simulate weak Cooper elimination, CutSat++ uses a total order ≺ over all variables such that x k ≺ x j for all guarded variables x k and unguarded variables x j . 11 In relation to weak Cooper elimination, the order ≺ describes the elimination order for the unguarded variables, viz., x j

x i if x j is eliminated before x i . This also means that the maximal

M, C =⇒ CS M , C if                        (x j , C ) is a conflicting core, x j is unguarded, all x k ≺ x j are fixed and C ⊆ C, if J ∈ C is a conflict, then top(J) ≺ x j , w-cooper(x j , C ) = (R y , R c ), x i = min I∈Rc {top(I)}, M = prefix(M, x i ), C = C ∪ R y ∪ R c Solve-Div-Left M, C =⇒ CS M, C if                divisibility constraints I 1 , I 2 ∈ C,
x j is unguarded and top in I 1 and I 2 , all other variables in I 1 , I 2 are fixed,

(I 1 , I 2 ) = div-solve(x j , I 1 , I 2 ), C = (C \ {I 1 , I 2 }) ∪ {I 1 , I 2 }, I 2 is not a conflict Solve-Div-Right M, C =⇒ CS M , C if                        divisibility constraints I 1 , I 2 ∈ C,
x j is unguarded and top in I 1 and I 2 , all other variables in I 1 , I 2 are fixed,

{I 1 , I 2 } = div-solve(x j , {I 1 , I 2 }), C = (C \ {I 1 , I 2 }) ∪ {I 1 , I 2 }, I 2 is a conflict, x i = top(I 2 ), M = prefix(M, x i )
In the above rules, M = prefix(M, x i ) defines the largest prefix of M that contains only decided bounds for variables x k with x k ≺ x i . Figure 3.9: Our unguarded conflict resolution rules unguarded variable in a given subset of constraints C is the next variable that is eliminated from C by weak Cooper elimination. For this reason, unguarded conflict resolution also always targets the maximal unguarded variable in a given conflict set. Formally, we call a variable x j maximal in a constraint I if x j is contained in I, i.e., coeff(I, x j ) = 0, and all other variables contained in I are smaller, i.e., x k ≺ x j . Analogously, a variable x j is called maximal in a constraint set C if x j is contained in C, i.e., coeff(I, x j ) = 0 for an I ∈ C, and all other variables contained in C are smaller, i.e., x k ≺ x j . We also call the maximal variable in a constraint I (or a constraint set C ) its top variable and denote it by x j = top(I) (x j = top(C )).

In contrast to weak Cooper elimination, unguarded conflict resolution does not eliminate any variables but adds (unguarded) resolvents to the constraint set so that the satisfiability depends only on smaller variables, which essentially simulates the elimination of a variable. Moreover, unguarded conflict resolution does not add resolvents for all constraints in our problem, but only to so-called conflicting cores, i.e., a compact representation of the constraints that are responsible for a given conflict. Definition 3.5.1 (Conflicting Cores). Let S = M, C and C ⊆ C. Let x j = top(C ) be unguarded, let all other variables in C be fixed, let a, b > 0, and let l j = bound(-a j x j + p j ≤ 0, x j , ≥, M ) and u j = bound(b j x jq j ≤ 0, x j , ≤, M ). The pair (x, C ) is a conflicting core if it is of one of the following three forms:

(1) C = {-a j x j + p j ≤ 0, b j x jq j ≤ 0} and l j > u j , i.e., the lower bound from -a j x j + p j ≤ 0 contradicts the upper bound from b j x jq j ≤ 0; in this case, (x j , C ) is called an interval conflicting core and its unguarded resolvent is ({-y i ≤ 0, y ia j + 1 ≤ 0}, {b j p ja j q j + b j y i ≤ 0,

a j | y i + p j }) (2) C = {-a j x j + p j ≤ 0, b j x j -q j ≤ 0, d | c j x j + s j }
and l j ≤ u j , and for all v j ∈ [l j , u j ] ∩ Z we have d c j v j + L(s, M ), i.e., there exists no value for x j within the bounds defined by the two inequalities such that the divisibility constraint becomes satisfiable; in this case, (x j , C ) is called a divisibility conflicting core and its unguarded resolvent is ({-y i ≤ 0, y im ≤ 0}, {b j p ja j q j + b j y i ≤ 0, a j | y i + p j , a j d | c j p j + a j s j + c j y i }) (3) C = {d | c j x j + s j } and for all v j ∈ Z we have d cv j + L(s j , M ), i.e., there exists no integer value for x j that could satisfy d | c j x j + s j ; in this case (x j , C ) is called a diophantine conflicting core and its unguarded resolvent is (∅, {gcd(c j , d) | s j }).

In the first two cases, y i is a fresh variable and m = lcm a j ,

a j d gcd(a j d,c j ) -1.
We refer to the respective unguarded resolvents for a conflicting core (x j , C ) by the function w-cooper(x j , C ), which returns a pair (R y , R c ) as defined above. In relation to weak Cooper elimination, R y ∪ R c is the (potentially simplified) result of elimination x j from C with weak Cooper elimination. Note also that the newly introduced variable y i in each resolvent is guarded by the constraints in R y , which means y i must be placed in the order ≺ so it is smaller than all unguarded variables x k . Therefore, the resolvent R y ∪ R c implies the satisfiability of ∃x j ∈ Z.C although it contains only smaller variables than x j .

Out of convenience, we are also introducing some new terminology with regard to conflicting cores: Firstly, a variable x j is called a conflicting variable if there is a conflicting core (x j , C ) in some state S. Secondly, a pair (x j , C ) is a potential conflicting core if there exists a state S where (x j , C ) is a conflicting core.

Next, we define a generalization of unguarded resolvents. We do so because the unguarded resolvents generated out of conflicting cores will be further processed by CutSat++ and we must guarantee that the processed unguarded resolvents still imply the satisfiability of the conflicting core constraints. Moreover, this generalization guarantees that CutSat++ introduces only one unguarded resolvent for every conflicting core. Definition 3.5.2 (Unguarded Resolvents). A set of constraints R is an unguarded resolvent for (x j , C ) if R → ∃x j ∈ Z.C and ∀J ∈ R. top(J) ≺ x j , i.e., the satisfiability of the unguarded resolvent R implies the satisfiability of C , while using only variables that are smaller than x j .

All unguarded resolvents of Definition 3.5.1 are also unguarded resolvents in the sense of the below definition (see also end of Section 3.4).

Lemma 3.5.3 (Unguarded Resolvent Soundness). Let C be a subset of C (C ⊆ C). Let w-cooper(x j , C ) = (R y , R c ). Let R = R y ∪ R c . Then ∃y i ∈ Z.C ∪ R ≡ C. Furthermore, R is an unguarded resolvent for (x j , C ).
Proof. The interval conflicting core is the only new case; however, the two Lemmas 3.4.4 and 3.4.5 are still enough to prove the soundness of all three cases because w-cooper(x j , {-a j x j + p j ≤ 0, b j x jq j ≤ 0}) is equivalent to w-cooper(x j , {-a j x j + p j ≤ 0, b j x jq j ≤ 0, 1 | x j }) (after simplifications). Therefore, R → ∃x j ∈ Z.C due to Lemmas 3.4.4 and 3.4.5. Finally, top(J) ≺ x j holds for all J ∈ R because y i is guarded and therefore smaller than x j and all other variables in R appear in C where x j is maximal.

Let us now take a closer look at the rules that make up unguarded conflict resolution and their many restrictions/conditions. The first rule is Resolve-Weak-Cooper and it (i) adds an unguarded resolvent (R y , R c ) for a given conflicting core (x j , C ) and (ii) backtracks to a prefix of the model in which (x j , C ) is no longer a conflicting core. We restrict the rule Resolve-Weak-Cooper (Figure 3.9) to unguarded constraints because we could otherwise generate infinitely many guarded variables. This poses no problem for efficiency because the standard conflict resolution is already very efficient on guarded constraints. Moreover, Resolve-Weak-Cooper requires that the conflicting variable x j of the conflicting core (x, C ) is the top variable in the constraints of C . This simulates a setting where all variables x i with x j ≺ x i are already eliminated. The restrictions also prevent Resolve-Weak-Cooper from being applied to the conflicting core (x j , C ) whenever our set of constraints already contains an unguarded resolvent R for this core. Since Resolve-Weak-Cooper adds an unguarded resolvent for every conflicting core it is applied to, this also means that Resolve-Weak-Cooper is never applied twice to the same conflicting core. Lemma 3.5.4 (Blocking Resolve-Weak-Cooper). Let S = M, C be a Cut-Sat++ state. Let C ⊆ C and x 1 be an unguarded variable. Let R ⊆ C be an unguarded resolvent for (x j , C ). Then Resolve-Weak-Cooper is not applicable to (x j , C ).

Proof. Assume for contradiction that D = (x j , C ) is a conflicting core, R ∈ C is an unguarded resolvent for D in state S, and Resolve-Weak-Cooper is applicable to D in state S. Resolve-Weak-Cooper requires that all variables x i ≺ x j are fixed (Figure 3.9). This holds especially for all variables in R (Definition 3.5.2). Due to the restriction in Resolve-Weak-Cooper that every conflict J ∈ C has top(J) ≺ top(I), there is no conflict in R. Furthermore, since all variables x i ≺ x j are fixed, R is satisfied by the partial assignment defined by M . By Definition 3.5.1, all conflicting cores have no satisfiable solution for x j under partial model M . However, by Definition 3.5.2, R satisfiable implies that there exists a value for x j such that C is satisfiable under M . This contradicts the assumption that (x j , C ) is a conflicting core! If we add unguarded resolvents again and again, then CutSat++ will reach a state after which every encounter of a conflicting core guarantees a conflict in a guarded constraint. From this point forward, CutSat++ will not apply Resolve-Weak-Cooper. The remaining guarded conflicts are resolved with the rules Conflict and Conflict-Div. However, this works only because of two other facets of unguarded conflict resolution: the eager toplevel propagating strategy and the two rules Solve-Div-Left and Solve-Div-Right. Both facets are needed so CutSat++ can compact any unguarded conflict into a conflicting core.

The rules Solve-Div-Left and Solve-Div-Right (Figure 3.9) combine divisibility constraints with the function div-solve(x j , {I 1 , I 2 }) as done a priori to weak Cooper elimination. We need these rules because our conflicting cores contain at most one divisibility constraint at a time. This could be problematic (without Solve-Div-Left and Solve-Div-Right) because there are unguarded conflicts that are only conflicting because of multiple divisibility constraints:

Example 3.5.5. Let C := {x 1 ≤ 1 I 1 , 2 | x 2 + x 1 J 1 , 2 | x 2 J 2
} be a problem, let the variable order be given by x 1 ≺ x 2 , and let our partial model be

[[x 1 ≤ I 1 1, x 1 ≥ 1]].
Then J 1 and J 2 are alone satisfiable but together they are a conflict for x 2 because x 2 can never be even (2 | x 2 ) and odd (2 | x 2 + 1) at the same time. Moreover, we cannot resolve this problem with Resolve-Weak-Cooper because there are currently no conflicting cores.

97

However, if we apply div-solve(x 2 , {J 1 , J 2 }) = {J 1 , J 2 } to combine J 1 and J 2 and replace {J 1 , J 2 } with {J 1 , J 2 }, then our constraint set transforms to:

C := {x 1 ≤ 1 I 1 , 4 | 2x 2 + 2x 1 J 1 , 2 | x 1 J 2 },
in which J 2 is a divisibility constraint that is a conflict on its own.

In fact, CutSat++ can always combine conflicts over multiple divisibility constraints using the rules Solve-Div-Left and Solve-Div-Right. We split this combination into two rules because div-solve(x j , {I 1 , I 2 }) = {I 1 , I 2 } might move a conflict over the variable x j into a conflict over a smaller variable x i , i.e., x i ≺ x j . In this case, we prefer to backtrack (with the rule Solve-Div-Right) to a prefix of our model where we can use the newly combined constraints {I 1 , I 2 } to prevent the construction of our conflicting model.

The other restrictions/conditions in Solve-Div-Left and Solve-Div-Right match restrictions of Resolve-Weak-Cooper. We again restrict the application of div-solve(x j , {I 1 , I 2 }) to constraints where x j is the top variable and where all variables x i in I 1 and I 2 with x i = x j are fixed. As before, this ordering restriction simulates the order of elimination, i.e., we apply div-solve(x j , {I 1 , I 2 }) in a (simulated) setting where all variables x i with x j ≺ x i appear to be eliminated in I 1 and I 2 . Otherwise, divergence is possible (see Example 3.3.6). Requiring smaller variables x k to be fixed prevents the accidental generation of a conflict for an unguarded variable x k by div-solve(x j , {I 1 , I 2 }).

As mentioned before, Solve-Div-Left and Solve-Div-Right alone do not yet guarantee that CutSat++ is able to compact all unguarded constraints into conflicting cores. We also need an eager top-level propagating strategy, as defined below: Definition 3.5.6 (Eager Top-Level Propagating Strategy). We call a strategy for CutSat++ eager top-level propagating if we restrict propagations and decisions for every state M, C in the following way:

1. Let x j be an unguarded variable. Let ∈ {≤, ≥}. Then we only allow to propagate bounds x j bound(I, (c) either L(x j , M ) = -∞ and U(x j , M ) = ∞ or there exists at most one divisibility constraint in C with x j as top variable.

x j , , M ) if x j is the top variable in I. Moreover, if I is a divisibility constraint d | a j x j + p j , then we only propagate d | a j x j + p j if: (a) L(x j , M ) = -∞ and U(x j , M ) = ∞ or (b) gcd(a j , d) | L(p j ,
An eager top-level propagating strategy has two advantages. First, the strategy dictates an order of influence over the variables, i.e., a bound for unguarded variable x j is influenced only by previously propagated bounds for variables x i with x i ≺ x j . This in itself already prevents divergence due to bound propagation. Moreover, the strategy makes decisions for unguarded variable x j only when all constraints with x j = top(I) are fixed and satisfied by the decision. This means, any conflict I ∈ C with x j = top(I) is impossible as long as the decision for x j remains on the bound sequence. We need this restriction because Resolve-Weak-Cooper can only resolve conflicts between constraints (see Definition 3.5.1) and, therefore, cannot resolve conflicts based on top variable decisions. For the same purpose, i.e., avoiding conflicts I where x j = top(I) is fixed by a decision, CutSat++ backjumps in the rules Resolve-Weak-Cooper and Solve-Div-Right to a state where this is not the case. To avoid stuck states resulting from the eager top-level propagating strategy, the slack variable x S has to be the smallest unguarded variable in ≺. Otherwise, the constraints x jx S ≤ 0, -x jx S ≤ 0 introduced by Slack-Intro cannot be used to propagate bounds for variable x j and x j would remain stuck. Definition 3.5.7 (Strictly-Two-Layered Strategy). A strategy is strictlytwo-layered if:

(1) it is reasonable (Definition 3.2.2), (2) it is eager top-level propagating, (3) the Forget, Conflict, Conflict-Div rules only apply to guarded constraints, (4) Forget cannot be applied to a divisibility constraint or a constraint contained in an unguarded resolvent, and (5) only guarded constraints are used to propagate guarded variables.

The above strictly-two-layered strategy is the final restriction to Cut-Sat++. With the condition 3.5.7-(3) it partitions conflict resolution into two layers: While every unguarded conflict is handled with the rules Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right (Figure 3.9), every guarded conflict is handled with the rules Conflict(-Div). The conditions 3.5.7-(1) and 3.5.7-(5) make the guarded variables independent from the unguarded variables. The condition 3.5.7-(2) prevents bound propagation and guarantees that all unguarded conflicts can be compacted into conflicting cores. The condition 3.5.7-(4) guarantees that no unguarded resolvent is ever removed (just transformed) and that all conflicting cores stay blocked after one application of Resolve-Weak-Cooper. We assume for the remainder of the paper that all runs of CutSat++ follow a strictly-two-layered strategy.

To better show how CutSat++ works, let us look again at the examples from Section 3.2. In contrast to CutSat, CutSat++ terminates with a correct solution on these examples:

[[]], C =⇒ Propagate CS [[x 1 ≥ I 1 0]], C =⇒ Propagate CS [[x 1 ≥ I 1 0, x 2 ≥ I 2 0]], C =⇒ Propagate CS [[x 1 ≥ I 1 0, x 2 ≥ I 2 0, x 3 ≤ I 3 0]], C =⇒ Propagate CS [[x 1 ≥ I 1 0, x 2 ≥ I 2 0, x 3 ≤ I 3 0, x 3 ≥ I 4 0]], C =⇒ Conflict CS [[x 1 ≥ I 1 0, x 2 ≥ I 2 0, x 3 ≤ I 3 0, x 3 ≥ I 4 0]], C, x 3 + 1 ≤ 0 =⇒ Resolve CS [[x 1 ≥ I 1 0, x 2 ≥ I 2 0, x 3 ≤ I 3 0]], C, 1 ≤ 0 =⇒ Unsat CS unsat
:= { -x 1 ≤ 0 I 1 , -x 2 ≤ 0 I 2 , -x 3 ≤ 0 I 3 , x 3 ≤ 0 I 4 , x 3 + 1 ≤ 0 I 5 , 1 -x 1 + x 2 ≤ 0 J 1 , x 1 -x 2 ≤ 0 J 2 }
be a problem. Let the variable order be given by x 3 ≺ x 2 ≺ x 1 . Then CutSat++ would solve the problem as depicted in Figure 3.10. First of all, CutSat++ has to propagate I 1 , I 2 , I 3 , and I 4 , i.e., all constraints containing only one variable. However, this already turns I 5 into a conflicting constraint. Although we could add decided bounds and propagated bounds until (x 1 , {J 1 , J 2 }) turns into a conflicting core, we could never apply Resolve-Weak-Cooper to (x 1 , {J 1 , J 2 }) because top(I 5 ) ≺ x 1 and I 5 is conflicting. Therefore, CutSat++ has to apply at some point Conflict to I 5 .

After applying Resolve to x 3 + 1 ≤ 0 with the bound x 3 ≥ I 4 0, CutSat++ has derived the trivially unsatisfiable constraint 1 ≤ 0. Finally, CutSat++ uses 1 ≤ 0 to apply Unsat and, thereby, return unsat.

Example 3.5.9.

Let C 0 = {4 | 2x 1 + 2x 2 , 2 | x 1 + x 3 } be a problem.
Let the variable order be given by x 1 ≺ x 2 ≺ x 3 . Then CutSat++ would solve the problem as depicted in Figure 3.11. Since C 0 has no constraints containing only one variable, CutSat++ cannot propagate any bounds and all variables are stuck in state S 0 . To resolve the stuck state, CutSat++ applies Slack-Intro to x 1 . Now, CutSat++ is able to propagate the bound x S ≥ Ix S 0 for the slack variable x S and even fix it with the decided bound x S ≤ 0. Since x S is fixed, CutSat++ propagates the bounds x 1 ≤ I 1 0 and x 1 ≥ I 2 0. However, the variables x 2 and x 3 are again stuck. We resolve this by repeating what we did for x 1 , first to the variable x 2 and then to the

[[]], C 0 =⇒ Slack-Intro CS [[]], C 1
where

C 1 := C 0 ∪ {-x S ≤ 0 Ix S , x 1 -x S ≤ 0 I 1 , -x 1 -x S ≤ 0 I 2 } =⇒ Propagate CS [[x S ≥ Ix S 0]], C 1 =⇒ Decide CS [[x S ≥ Ix S 0, x S ≤ 0]], C 1 =⇒ Propagate CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0]], C 1 =⇒ Propagate CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0]], C 1 =⇒ Slack-Intro CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0]], C 2 where C 2 := C 1 ∪ {x 2 -x S ≤ 0 I 3 , -x 2 -x S ≤ 0 I 4 } =⇒ Propagate CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0, x 2 ≤ I 3 0]], C 2 =⇒ Propagate CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0, x 2 ≤ I 3 0, x 2 ≥ I 4 0]], C 2 =⇒ Slack-Intro CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0, x 2 ≤ I 3 0, x 2 ≥ I 4 0]], C 3 where C 3 := C 2 ∪ {x 3 -x S ≤ 0 I 5 , -x 3 -x S ≤ 0 I 6 } =⇒ Propagate CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0, x 2 ≤ I 3 0, x 2 ≥ I 4 0, x 3 ≤ I 5 0]], C 3 =⇒ Propagate CS [[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0, x 2 ≤ I 3 0, x 2 ≥ I 4 0, x 3 ≤ I 5 0, x 3 ≥ I 6 0]], C 3 =⇒ Sat CS ν[[[x S ≥ Ix S 0, x S ≤ 0, x 1 ≤ I 1 0, x 1 ≥ I 2 0, x 2 ≤ I 3 0, x 2 ≥ I 4 0, x 3 ≤ I 5 0, x 3 ≥ I 6 0]]], C 3
:= { -x 1 ≤ 0 I 1 , -x 2 ≤ 0 I 2 , -x 3 ≤ 0 I 3 , x 3 ≤ 0 I 4 , 1 -x 1 + x 2 + x 3 ≤ 0 J 1 , x 1 -x 2 -x 3 ≤ 0 J 2 } be a problem. Let (R y , R c ) be the output of w-cooper(y 1 , {J 1 , J 2 }) so that (R y , R c ) := ({-y 1 ≤ 0 K 1 , y 1 ≤ 0 K 2 }), {y 1 + 1 ≤ 0 K 3 , 1 | y 1 + x 2 + x 3 + 1 K 4 }).
Let the variable order be given by x 3 ≺ x 1 ≺ x 2 . Then CutSat++ would solve the problem as depicted in Figure 3.12. First of all, CutSat++ has

[[]], C 0 =⇒ Propagate CS [[x 3 ≥ I 3 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0, x 2 ≥ I 2 0]], C 0 =⇒ Decide CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0, x 2 ≥ I 2 0, x 1 ≤ 0]], C 0 =⇒ Resolve-Weak-Cooper CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0, x 2 ≥ I 2 0]], C 1
where the new order is to propagate I 3 , I 4 , I 1 , and I 2 , i.e., all constraints containing only one variable. Then, CutSat++ fixes x 1 with a decided bound x 1 ≤ 0, which turns (x 2 , {J 1 , J 2 }) into a conflicting core. Since only J 1 is a conflicting constraint and x 2 = top(J 1 ), the rule Resolve-Weak-Cooper is applicable and Cut-Sat++ will use it to resolve the conflicting core (x 2 , {J 1 , J 2 }). The resulting resolvent is (R y , R c ) for which CutSat++ introduces the fresh variable y 1 and updates the variable order to x 3 ≺ y 1 ≺ x 1 ≺ x 2 . Resolve-Weak-Cooper also truncates the bound sequence to remove all decided bounds for variables greater than or equal to y 1 . Next, CutSat++ has to propagate the newly introduced constraints K 1 and K 2 because CutSat++ follows a reasonable strategy and K 1 and K 2 are of the form ±y 1 +c ≤ 0. However, this turns the constraint K 3 into a guarded conflict constraint. After applying Conflict to K 3 and resolving K 3 with x 1 ≥ K 1 0, CutSat++ has derived the trivially unsatisfiable constraint 1 ≤ 0. Finally, CutSat++ uses 1 ≤ 0 to apply Unsat and, thereby, return unsat.

x 3 ≺ y 1 ≺ x 1 ≺ x 2 , and C 1 := C 0 ∪ R y ∪ R c =⇒ Propagate CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0, x 2 ≥ I 2 0, y 1 ≤ K 2 0]], C 1 =⇒ Propagate CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0, x 2 ≥ I 2 0, y 1 ≤ K 2 0, y 1 ≥ K 1 0]], C 1 =⇒ Conflict CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0, x 2 ≥ I 2 0, y 1 ≤ K 2 0, y 1 ≥ K 1 0]], C 1 , y 1 + 1 ≤ 0 =⇒ Resolve CS [[x 3 ≥ I 3 0, x 3 ≤ I 4 0, x 1 ≥ I 1 0, x 2 ≥ I 2 0, y 1 ≤ K 2 0]], C 1 , 1 ≤ 0 =⇒ Unsat CS unsat
I 1 , x 3 ≤ 0 I 2 , -x 4 ≤ 0 I 3 , x 4 ≤ 0 I 4 , -x 5 ≤ 0 I 5 , -x 5 -x 1 -x 3 ≤ 0 I 6 , -x 5 + x 1 + x 3 ≤ 0 I 7 , -x 5 -x 2 -x 4 ≤ 0 I 8 , -x 5 + x 2 + x 4 ≤ 0 I 9 , 2 + 3x 1 -4x 2 + 3x 3 -4x 4 ≤ 0 J 1 , -1 + 3x 1 -4x 2 + 3x 3 -4x 4 ≤ 0 J 2 , -1 -3x 1 + 2x 2 -3x 3 + 2x 4 ≤ 0 J 3 } be a problem. Let C 1 := C 0 ∪ R y 1 ∪ R 1 , where (R y 1 , R 1 ) := ( {-y 1 ≤ 0 K 1 , -3 + y 1 ≤ 0 K 2 }), {4 | 2 + 3x 1 + 3x 3 + y 1 K 3 , -3x 1 -3x 3 + y 1 ≤ 0 K 4 }) is the output of w-cooper(x 2 , {J 1 , J 3 }). Let C 2 := C 1 ∪ R y 2 ∪ R 2 , where (R y 2 , R 2 ) := ({-y 2 ≤ 0 K 5 , -2 + y 2 ≤ 0 K 6 }), {3 | y 1 + y 2 K 7 , y 1 -3x 5 + y 2 ≤ 0 K 8 }) is the output of w-cooper(x 1 , {K 4 , I 7 }). Let C 3 := C 2 ∪ R y 3 ∪ R 3 , where (R y 3 , R 3 ) := ( {-y 3 ≤ 0 K 9 , -11 + y 3 ≤ 0 K 10 }), {3 | y 1 + y 3 K 11 , 4 | 2 + 2y 1 + y 3 K 12 , y 1 -3x 5 + y 3 ≤ 0 K 13 })
is the output of w-cooper(x 1 , {K 3 , K 4 , I 7 }). 12 Let the variable order be given by x 3 ≺ x 4 ≺ x 5 ≺ x 1 ≺ x 2 . Then CutSat++ would solve the problem as depicted in Figures 3.13 First of all, CutSat++ has to propagate I 1 , I 2 , I 3 , I 4 , and I 5 , i.e., all constraints containing only one variable. In order to propagate bounds for x 1 and x 2 , CutSat++ has to fix x 5 first. It does so with the decided bound x 5 ≤ 0. Then, CutSat++ propagates the constraints I 6 , I 7 , I 8 , and I 9 , which fixes both x 1 and x 2 to 0 and turns (x 2 , {J 1 , J 3 }) into a conflicting core. Since only J 1 is a conflicting constraint and x 2 = top(J 1 ), the rule Resolve-Weak-Cooper is applicable and CutSat++ will use it to resolve the conflicting core (x 2 , {J 1 , J 3 }). The resulting resolvent is (R y 1 , R 1 ) for which CutSat++ introduces the fresh variable y 1 and updates the variable order to

x 3 ≺ x 4 ≺ y 1 ≺ x 5 ≺ x 1 ≺ x 2 .
Resolve-Weak-Cooper does not actually truncate the current bound sequence because the current bound sequence is already its own largest prefix that contains only decided bounds for variables x j with x j ≺ x 1 . Although J 1 is still a conflict in the re-

[[]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0]], C 0 =⇒ Decide CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0, x 1 ≤ I 7 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0, x 1 ≤ I 7 0, x 2 ≥ I 8 0]], C 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0, x 1 ≤ I 7 0, x 2 ≥ I 8 0, x 2 ≤ I 9 0]], C 0 =⇒ Resolve-Weak-Cooper CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0, x 1 ≤ I 7 0, x 2 ≥ I 8 0, x 2 ≤ I 9 0]], C 1
where the new order is

x 3 ≺ x 4 ≺ y 1 ≺ x 5 ≺ x 1 ≺ x 2 ,
and

C 1 := C 0 ∪ R y 1 ∪ R 1 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0, x 1 ≤ I 7 0, x 2 ≥ I 8 0, x 2 ≤ I 9 0, y 1 ≥ K 1 0]], C 1 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0, x 1 ≤ I 7 0, x 2 ≥ I 8 0, x 2 ≤ I 9 0, y 1 ≥ K 1 0, y 1 ≤ K 2 3]], C 1 =⇒ Decide CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, x 5 ≤ 0, x 1 ≥ I 6 0, x 1 ≤ I 7 0, x 2 ≥ I 8 0, x 2 ≤ I 9 0, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 1 ≥ 3]], C 1 =⇒ Resolve-Weak-Cooper CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0]], C 2
where the new order is

x 3 ≺ x 4 ≺ y 2 ≺ y 1 ≺ x 5 ≺ x 1 ≺ x 2 ,
and

C 2 := C 1 ∪ R y 2 ∪ R 2
CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0]], C 2 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2]], C 2 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0]], C 2 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3]], C 2 =⇒ Decide CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2]], C 2 =⇒ Propagate-Div CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1]], C 2
where S = M, C 2 is the state before the application of Propagate-Div, and

K 7 := div-derive(K 7 , y 1 , ≤, M ) = y 1 + y 2 -3 ≤ 0 =⇒ Propagate-Div CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1]], C 2
where S = M, C 2 is the state before the application of Propagate-Div, and

K 7 := div-derive(K 7 , y 1 , ≥, M ) = -y 1 -2y 2 -3 ≤ 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1]], C 2
where S = M, C 2 is the state before the application of Propagate, and

K 8 := tight(K 8 , x 5 , M ) = -x 5 + y 2 -1 ≤ 0 =⇒ Decide CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, x 5 ≤ 1]], C 2 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, x 5 ≤ 1, x 1 ≤ I 7 1]], C 2 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, x 5 ≤ 1, x 1 ≤ I 7 1, x 1 ≥ K 4 1]], C 2
where S = M, C 2 is the state before the application of Propagate, and

K 4 := tight(K 4 , x 1 , M ) = -x 1 -x 3 + y 2 -1 ≤ 0
[[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8, x 5 ≥ K 13 3, x 5 ≤ 3, x 1 ≤ I 7 3, x 1 ≥ K 4 1]], C 3
where S = M, C 3 is the state before the application of Propagate, and

K 4 := tight(K 4 , x 1 , M ) = -x 1 -x 3 + y 2 -1 ≤ 0 =⇒ Propagate-Div CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8, x 5 ≥ K 13 3, x 5 ≤ 3, x 1 ≤ I 7 3, x 1 ≥ K 4 1, x 1 ≥ K 3 2]], C 3
where S = M, C 3 is the state before the application of Propagate-Div, and

K 3 := div-derive(K 3 , x 1 , ≥, M ) = -x 1 -x 3 + 3y 2 -4 ≤ 0 =⇒ Propagate-Div CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8, x 5 ≥ K 13 3, x 5 ≤ 3, x 1 ≤ I 7 3, x 1 ≥ K 4 1, x ≥ K 3 2, x 1 ≥ K 3 3]], C 3
where S = M, C 3 is the state before the application of Propagate-Div, and

K 3 := div-derive(K 3 , x 1 , ≥, M ) = -x 1 -x 3 + 5y 2 -7 ≤ 0 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8, x 5 ≥ K 13 3, x 5 ≤ 3, x 1 ≤ I 7 3, x 1 ≥ K 4 1, x ≥ K 3 2, x 1 ≥ K 3 3, x 2 ≤ I 9 3]], C 3 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8, x 5 ≥ K 13 3, x 5 ≤ 3, x 1 ≤ I 7 3, x 1 ≥ K 4 1, x ≥ K 3 2, x 1 ≥ K 3 3, x 2 ≤ I 9 3, x 2 ≥ J 1 3]], C 3
where S = M, C 3 is the state before the application of Propagate, and ) for which Cut-Sat++ introduces the fresh variable y 3 and updates the variable order to

J 1 := tight(J 1 , x 2 , ≥, M ) = -x 2 -x 4 + 4y 2 -5 ≤ 0 =⇒ Sat CS ν[[[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8, x 5 ≥ K 13 3, x 5 ≤ 3, x 1 ≤ I 7 3, x 1 ≥ K 4 1, x ≥ K 3 2, x 1 ≥ K 3 3, x 2 ≤ I 9 3, x 2 ≥ J 1 3]]],
x 3 ≺ x 4 ≺ y 3 ≺ y 2 ≺ y 1 ≺ x 5 ≺ x 1 ≺ x 2 .
Resolve-Weak-Cooper also truncates the bound sequence to the largest prefix that contains only decided bounds for variables x j with x j ≺ y 1 . Again, CutSat++ has to first propagate all constraints containing only one variable, so K 9 and K 10 . Using the divisibility constraint K 12 , CutSat++ is able to refine the lower bound of y 3 to 8. Since CutSat++ cannot propagate any further bounds for y 3 , it fixes it with a decided bound to 8. Then, CutSat++ uses the inequality K 13 and the fixed variables y 1 and y 3 , to propagate 3 as the new lower bound for x 5 . With a decided bound, CutSat++ fixes x 5 to 3. By propagating the constraints I 7 , K 4 , and K 3 , CutSat++ fixes also x 1 to 3. Similarly, CutSat++ fixes x 2 to 3 by propagating I 9 and J 1 . In the resulting bound sequence all variables are fixed and the assignment is satisfying all constraints in C 3 . Therefore, CutSat++ ends with an application of Sat and returns the assignment: where

x 3 → 0, x 4 → 0, y 3 → 8, y 2 → 2, y 1 → 1, x 5 → 3, x 1 → 3, x 2 → 3 CS [[x 1 ≥ I 1 0]], C 0 =⇒ Propagate CS [[x 1 ≥ I 1 0, x 1 ≤ I 2 1]], C 0 =⇒ Propagate CS [[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0]], C 0 =⇒ Decide CS [[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0, x 1 ≥ 1]], C 0 =⇒ Resolve-Weak-Cooper CS [[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0]], C 1 where C 1 := C 0 ∪ J 2 =⇒ Propagate-Div CS [[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0, x 1 ≤ J 2 0]], C 1 where J 2 := div-derive(J 2 , x 1 , ≤, [[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0]]) = x 1 ≤ 0 =⇒ Decide CS [[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0, x 1 ≤ J 2 0, x 2 ≤ 0]], C 1 =⇒ Sat CS ν[[[x 1 ≥ I 1 0, x 1 ≤ I 2 1, x 2 ≥ I 3 0, x 1 ≤ J 2 0, x 2 ≤ 0]]], C 1
J 2 = 2 | x 1 .
Let the variable order be given by x 1 ≺ x 2 . Then CutSat++ would solve the problem as depicted in Figure 3.17. First of all, CutSat++ has to propagate I 1 , I 2 , and I 3 , i.e., all constraints containing only one variable. Then CutSat++ will fix x 1 to the bound x 1 ≤ I 1 1 with the decided bound x 1 ≥ 1, which turns (x 2 , {J}) into a conflicting core. Note that CutSat++ cannot use J 1 = 6 | 4x 2 + x 1 to propagate a bound for x 2 because CutSat++ follows an eager top-level propagating strategy and gcd(4, 6) = 2 does not divide 1 the value x 1 is fixed to. For the same reason CutSat++ cannot fix x 2 with a decided bound. Since only J is a conflicting constraint and x 2 = top(J), the rule Resolve-Weak-Cooper is applicable and CutSat++ will use it to resolve the conflicting core (x 2 , {J}). The resulting resolvent is (∅, {J 2 }), where J 2 = 2 | x 1 . Resolve-Weak-Cooper also truncates the bound sequence so that all decided bounds for variables greater than or equal to x 1 are removed. Now CutSat++ propagates the constraint J 2 to fix x 1 to 0 instead of 1. Since 6 | 4x 2 + x 1 is satisfied by the bounds for x 1 and x 2 , CutSat++ is also able to fix x 2 with a decided bound. Finally, CutSat++ returns the satisfiable assignment with an application of the Sat rule.

Termination and Completeness

The CutSat++ rules are Propagate, Propagate-Div, Decide, Sat, Unsat, Unsat-Div, Conflict, Conflict-Div, Resolve, Skip-Decision, Backjump, Slack-Intro, Learn, and Forget from CutSat g [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], as well as Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right (Figure 3.9). Before we prove termination and completeness for CutSat++, we have to prove another property over unguarded resolvents. We have proven in Section 3.5 that Resolve-Weak-Cooper applied to the conflicting core (x j , C ) adds an unguarded resolvent R that blocks another application of Resolve-Weak-Cooper to (x j , C ). However, CutSat++ is able to remove constraints from R with the rules Solve-Div-Left and Solve-Div-Right. This removes the original conflicting core R from our state. Nonetheless, CutSat++ is still unable to apply Resolve-Weak-Cooper to conflicting core (x j , C ) because the rules Solve-Div-Left and Solve-Div-Right guarantee that a new unguarded resolvent R for conflicting core (x j , C ) is introduced: Proof. Assume for a contradiction that C contains an unguarded resolvent R for (x j , C ) and C contains no unguarded resolvent R for (x j , C ). W.l.o.g., we assume that S is the first state after S where R C . By Definition 3.5.7-(4), CutSat++ with a strictly-two-layered strategy cannot remove constraints from an unguarded resolvent R except with the rules Solve-Div-Right and Solve-Div-Left. Through the equivalence proven for div-solve(x j , {I 1 , I 2 }) [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], we know that there exist

I 1 , I 2 ∈ C such that {I 1 , I 2 } ≡ {I 1 , I 2 } and R ⊆ (C \ {I 1 , I 2 }) ∪ {I 1 , I 2 }. We use this equivalence to construct R = (R \ {I 1 , I 2 }) ∪ {I 1 , I 2 } such that R → R. Since R → R and R → ∃x j ∈ Z.C , R is also an unguarded resolvent of (x j , C ) such that R → ∃x j ∈ Z.C . Furthermore, R is a subset of C ,
which contradicts our initial assumption! Together with Lemma 3.5.4, this property implies that Resolve-Weak-Cooper is applied at most once to every conflicting core encountered by CutSat++. This is essential for our termination proof.

Termination

For the termination proof of CutSat++, we consider a (possibly infinite) sequence of rule applications [[]], C 0 = S 0 =⇒ CS S 1 =⇒ CS . . . on a problem C 0 following the strictly-two-layered strategy. We will show that each such sequence of rule applications can be divided into five phases-the slacking phase, the unguarded resolution phase, the guarded search phase, the unguarded extension phase, and the end phase-and that each of those phases terminates after finitely many rule applications.

First, this sequence reaches a state S s (s ∈ N + 0 ), after a finite derivation of rule applications S 0 =⇒ CS . . . =⇒ CS S s , such that there is no further application of the rules Slack-Intro and Forget after state S s :

Lemma 3.6.2 (Slacking Phase). Let [[]], C 0 = S 0 =⇒ CS S 1 =⇒ CS .
. . be a sequence of rule applications applied to a problem C 0 following the strictlytwo-layered strategy. Then the sequence reaches a state S s (s ∈ N + 0 ), after at most finitely many rule applications S 0 =⇒ CS . . . =⇒ CS S s , such that there is no further application of the rules Slack-Intro and Forget after state S s .

Proof. Such a state S s exists for two reasons: Firstly, the strictly-two-layered strategy employed by CutSat++ is also reasonable. The reasonable strategy explicitly forbids infinite applications of the rule Forget. Secondly, the Slack-Intro rule is applicable only to stuck variables and only once to each stuck variable. Only the initial set of variables can be stuck because all variables x j introduced by one of the rule applications are introduced with at least one constraint x jb j ≤ 0 that allows at least one propagation for the variable. Therefore, the rules Slack-Intro and Forget are at most finitely often applicable.

Next, the sequence reaches a state S w (w ≥ s), after a finite derivation of rule applications S s =⇒ CS . . . =⇒ CS S w , such that there is no further application of the rules Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right after state S w : The rules Slack-Intro, Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right are applicable only to unguarded constraints. Through the strictlytwo-layered strategy, they are also the only rules producing unguarded constraints. Therefore, they form a closed loop with respect to unguarded constraints, which we use in our termination proof. We have shown in the previous paragraph that S s =⇒ CS . . . =⇒ CS S w contains no application of the rule Slack-Intro. By Lemma 3.5.4, an application of Resolve-Weak-Cooper to the conflicting core (x j , C ) prevents any further applications of Resolve-Weak-Cooper to the same core. By Definition 3.5.1, the constraints learned through an application of Resolve-Weak-Cooper contain only variables x i such that x i ≺ x j . Therefore, an application of Resolve-Weak-Cooper blocks a conflicting core (x j , C ) and introduces potential conflicting cores only for smaller variables than x j . This strict decrease in the conflicting variables guarantees that we encounter only finitely many conflicting cores in unguarded variables. Therefore, Resolve-Weak-Cooper is applicable at most finitely often. An analogous argument applies to the rules Solve-Div-Left and Solve-Div-Right. Thus the rules Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right are applicable at most finitely often.

Lemma 3.6.3 (Unguarded Resolution Phase). Let [[]], C 0 = S 0 =⇒ CS S 1 =⇒ CS .
. . be a sequence of rule applications applied to a problem C 0 following the strictly-two-layered strategy. Then the sequence reaches a state S w , after finitely many rule applications S 0 =⇒ CS . . . =⇒ CS S w , such that there is no further application of the rules Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right after state S w .

Proof. By Lemma 3.6.2, we assume, w.l.o.g., that the sequence continues from a state S s such that S s is reached by the sequence after at most finitely many rule applications S 0 =⇒ CS . . . =⇒ CS S s and that there is no further application of the rules Slack-Intro and Forget after state S s . Let x 1 ≺ . . . ≺ x n be the order of variables for all unguarded variables x i . Next, we define a weight vector that strictly decreases after every call to Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right. For this weight vector, we define cores(x i , C) as the set of potential conflicting cores in the problem C with conflicting variable x i . We also define a subset woSR(x i , C) of cores(x i , C) so it contains all potential conflicting cores within cores(x i , C) that do not have an unguarded resolvent R ⊆ C. It is easy to see that | cores(x i , C)| ≤ 2 |C| and, therefore, both functions define finite sets. Now we define the weight vector W(S) for every state S = M, C, I :

W(S) = (| cores(x n , C)|, | woSR(x n , C)|, . . . , | cores(x 1 , C)|, | woSR(x 1 , C)|)
We order the two W vectors of two subsequent search-states with the wellfounded lexicographic order > lex based on the well-founded order >.

By Definition 3.5.7, Conflict(-Div) is only applicable to guarded constraints and guarded variables are only propagated through guarded constraints. Therefore, the conflict I in a conflict state M, C, I stays always guarded-even after an application of the Resolve rule-and Learn is only applicable to guarded constraints. Therefore, Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right are the only rules potentially learning unguarded constraints and, thereby, the only rules that can increase | cores(x i , C)| and | woSR(x i , C)| between two subsequent states S =⇒ CS S. After all other transitions S =⇒ CS S, where S = M , C and S = M, C it holds that W(S ) ≥ lex W(S).

The reasons why the weight vector strictly decreases, i.e., W(S ) > lex W(S), whenever CutSat++ applies Solve-Div-Left, Solve-Div-Right, or Resolve-Weak-Cooper are as follows:

1. By Lemma 3.5.4, an application of Resolve-Weak-Cooper to conflicting core (x i , C * ) implies that there is no unguarded resolvent R ⊆ C for (x i , C * ). By Lemma 3. 2. Let Solve-Div-Left (Solve-Div-Right) be applied to the pair of divisibility constraints (I 1 , I 2 ) such that top(I 1 ) = x i and div-solve

(x i , {I 1 , I 2 }) = {I 1 , I 2 }. The new constraint set is C = C \ {I 1 , I 2 } ∪ {I 1 , I 2 }.
The number of potential conflicting cores is independent of the actual divisibility constraints in C . Only the number of divisibility constraints with top(I) = x i is important. Therefore, removing I 1 and replacing it with I 1 doesn't increase the number of cores, i.e., | cores(

x i , C \ {I 1 } ∪ {I 1 })| = | cores(x i , C )|.
We will, however, decrease the number of conflicting cores in x i , i.e., | cores(x i , C )| > | cores(x i , C)|, because we replace I 2 with I 2 where top(I 2 ) ≺ x i . Since all variables x j in I 1 and I 2 are smaller than or equal to x i , we do not introduce any new conflicting cores for

x k with k > i; thus, | cores(x k , C )| = | cores(x k , C)| for k > i. Finally, Lemma 3.6.1 implies that | woSR(x k , C )| ≥ | woSR(x k , C)| for k > i. Therefore, W(C ) > lex W(C).
This proves already that the W vector monotonically decreases with respect to the order > lex if we continue from the before mentioned state S s . Moreover, the set of weight vectors has a minimum (0, . . . , 0) because no set can contain less than zero elements. As > lex is well-founded, there exists no way to decrease the weight W(C s ) without reaching the minimum (0, . . . , 0) after finitely many applications of the rules Solve-Div-Left, Solve-Div-Right, or Resolve-Weak-Cooper. Finally, the W vector cannot decrease below (0, . . . , 0); hence, CutSat++ is not able to apply Solve-Div-Left, Solve-Div-Right, or Resolve-Weak-Cooper after we reach a state S with W(S) = (0, . . . , 0). We conclude that the rules Solve-Div-Left, Solve-Div-Right, and Resolve-Weak-Cooper are at most finitely often applicable.

Next, the sequence reaches a state S g (g ≥ w), after a finite derivation of rule applications S w =⇒ CS . . . =⇒ CS S g , such that the bounds remain invariant for every guarded variable x i , i.e., L(x i , M g ) = L(x i , M j ) and U(x i , M g ) = U(x i , M j ) for every state S j = M j , C j , I j after S g = M g , C g , I g (j ≥ g). CutSat++ reaches such a state because the strictlytwo-layered strategy guarantees that unguarded variables do not influence guarded variables. Due to this independence, it is easy to extend the proof for the completely guarded case (see [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]) so it also proves that CutSat++ has to reach a state S g . By definition of S g , we now also know that the sequence after S g contains no further propagations, decisions, or conflict resolutions for the guarded variables. Lemma 3.6.4 (Guarded Search Phase). Let [[]], C 0 = S 0 =⇒ CS S 1 =⇒ CS . . . be a sequence of rule applications applied to a problem C 0 following the strictly-two-layered strategy. Then the sequence reaches a state S g , after finitely many rule applications S 0 =⇒ CS . . . =⇒ CS S g , such that the bounds remain invariant for every guarded variable x i .

Proof. This proof is based on the termination proof for CutSat on finite problems, i.e., problems without unguarded variables [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. The proof uses a weight function that strictly decreases whenever CutSat++ changes a bound for a guarded variable and otherwise stays the same. By Lemmas 3.6.2 and 3.6.3, we assume, w.l.o.g., that the sequence continues from a state S w such that S w is reached by the sequence after at most finitely many rule applications S 0 =⇒ CS . . . =⇒ CS S w , and there is no further application of the rules Slack-Intro, Forget, Resolve-Weak-Cooper, Solve-Div-Left, and Solve-Div-Right after state S w . The level B of a state S = M, C is the number of decisions for guarded variables in M . The maximal prefix of M containing only j decisions for guarded variables is denoted by B-subseq j (M ) = M j . Since CutSat++ follows a reasonable strategy, it prefers to propagate constraints of the form ±x kb k ≤ 0. This allows us to assume, w.l.o.g., that M 0 contains both a lower and upper bound for all guarded variables x i . The guarded weight of the j-th level B is defined by the function w B (M j ):

w B (M j ) = x i is guarded (U(x i , M j ) -L(x i , M j )) . The guarded weight of a state is the vector:

weight

B ( M, C ) = w B (B-subseq 0 (M )), • • • , w B (B-subseq n (M ))
, where n is the number of guarded variables. We order the two weight Bvectors of two subsequent search-states with the well-founded lexicographic order > lex based on the well-founded order >. It is easy to see that the minimum of weight B is (0, . . . , 0) and that any change to a bound of a guarded variable changes weight B . Furthermore, by the definition of the strictly-two-layered strategy, we see that we only propagate guarded variables with guarded constraints. Thus, the strategy also implies that the conflict rules-Conflict, Conflict-Div, Backjump, Resolve, Skip-Decision, Unsat, and Learn-only handle guarded constraints. Given the proof for Theorem 2 in [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], we see that every application of Propagate, Propagate-Div, and Decide applied to a guarded variable decreases weight B strictly. We see in the same proof that weight B strictly decreases between one application of Conflict(-Div) and Backjump as long as the conflict rules handle only guarded constraints-as is the case for CutSat++. Since the bound sequence M is finite, the conflict rules are at most |M | times applicable between one application of Conflict(-Div), and Backjump or Unsat. The remaining rules-Propagate, Propagate-Div, and Decide-applied to unguarded variables, have no influence on weight B or the bounds of guarded variables. Since weight B cannot decrease below (0, . . . , 0), we conclude that CutSat++ is not able to change the bounds for guarded variables infinitely often.

Next, the sequence reaches a state S u (u ≥ g), after a finite derivation of rule applications S g =⇒ CS . . . =⇒ CS S u , such that the bounds also remain invariant for every unguarded variable x i , i.e., L(x i , M u ) = L(x i , M j ) and U(x i , M u ) = U(x i , M j ) for every state S j = M j , C j , I j after S u = M u , C u , I u (j ≥ u). This is true because CutSat++ propagates and decides only unguarded variables after S g or ends with an application of Sat or Unsat(-Div). These claims are facts because CutSat++ employs a strictlytwo-layered strategy, which is also an eager top-level propagating strategy. Through the top variable restriction for propagating constraints, the eager top-level propagating strategy induces a strict order of propagation over the unguarded variables. Therefore, any bound for an unguarded variable x i is influenced only by bounds for variables x k ≺ x i . This strict variable order guarantees that unguarded variables are propagated and decided only finitely often.

Lemma 3.6.5 (Unguarded Extension Phase). Let [[]], C 0 = S 0 =⇒ CS S 1 =⇒ CS .
. . be a sequence of rule applications applied to a problem C 0 following the strictly-two-layered strategy. Then the sequence reaches a state S u , after finitely many rule applications S 0 =⇒ CS . . . =⇒ CS S u , such that the bounds remain invariant for every unguarded variable x i .

Proof. By Lemmas 3.6.2, 3.6.3, and 3.6.4, we assume, w.l.o.g., that the sequence continues from a state S g = M g , C g , I g such that S g is reached by the sequence after at most finitely many rule applications S 0 =⇒ CS . . . =⇒ CS S g and only the rules Sat, Unsat-Div, Propagate, Propagate-Div, and Decide are applied after S g , but only for unguarded variables. Assume for a contradiction that there exists an infinite CutSat++ run starting in S g . Since there is only a finite number of unguarded variables and no rule to undo a decision, the Decide rule is applied at most finitely often. Furthermore, any application of Sat or Unsat-Div ends the sequence making it finite. Thus, we assume, w.l.o.g., that there is no application to the rules Sat, Unsat-Div, and Decide in the infinite run starting in the state S g .

Since there are at most finitely many variables in state S g and no rule to introduce further variables after S g , there exists a smallest unguarded variable x i that is propagated infinitely often. We assume, w.l.o.g., that the run starting in S g propagates only variables x k bigger than or equal to x i . Therefore, the bounds of all variables x j smaller than x i remain invariant in all subsequent states S = M , C , I of S b , i.e., L(x j , M j ) = L(x j , M b ) and U(x j , M j ) = U(x j , M b ). Since there exists no applicable rule that changes the constraint set, we notice that the constraint set C g also remains invariant for all states after S g . Thus, we find all constraints C * that will be used to propagate x i in the set C g . Since CutSat++ is eager top-level propagating, any constraint I ∈ C * has x i as their top variable. This leads us to the deduction that bound(I, x i , , M ) = bound(I, x i , , M g ) for all subsequent states S = M , C , inequalities I ∈ C * , and ∈ {≤, ≥}. Since the bounds defined by the inequalities in C * remain invariant after state S g , CutSat++ propagates x i at most finitely often with inequalities.

Therefore, there exists an infinite CutSat++ run only if x i is propagated infinitely often with Propagate-Div. We assume, w.l.o.g., that the run starting in S g propagates x i only with Propagate-Div. Next, we deduce that variable x i stays unbounded in the remaining states of the derivation sequence. Otherwise, there exists a finite set x i ∈ {l i , . . . , u i } bounding x i and, therefore, allowing only finitely many propagations. In the case that x i stays unbounded, the definition of the eager top-level propagating strategy states that Propagate-Div is only applicable to

x i if I d = d | a i x i + p i ∈ C *
is the only divisibility constraint in C b with x i as their top variable. Furthermore, we know because of Definition 3.5.6 and Lemma 3.4.5 that there must exist

v i ∈ Z such that d | a i v i + L(p i , M g ) is satisfied
. Now we see that Propagate-Div propagates x i at most finitely often if we consider Lemma 3.2.1. More specifically, if the lower bound of x i is L(x, M g ) = l i = -∞ then Propagate-Div propagates for x i at most v il i lower bounds. If the upper bound of x i is U(x i , M g ) = u i = ∞ then Propagate-Div propagates for x i at most u iv i upper bounds. This contradicts the assumption that x i is the smallest variable propagated infinitely often, which in turn contradicts our initial assumption!

The final phase of our CutSat++ run is called the end phase. It is called so because the only rules applicable after state S u are the rules Sat, Unsat, and Unsat-Div, which lead to an end state. This is also the final reason why the sequence S 0 =⇒ CS S 1 =⇒ CS . . . must be finite. We conclude that CutSat++ always terminates: Theorem 3.6.6 (CutSat++ Termination). If CutSat++ starts from a start state [[]], C 0 , then there is no infinite derivation sequence.

Proof. By Lemmas 3.6.2, 3.6.3, 3.6.4, and 3.6.5, CutSat++ reaches a state S u after which only the rules Sat, Unsat, and Unsat-Div are applicable, which lead to an end state. Therefore, CutSat++ does not diverge.

Stuck States

Our CutSat++ calculus not only always terminates but it also never reaches a stuck state. Let x i be the smallest unfixed variable with respect to ≺. If x i is guarded then there always exist two constraints x iu i ≤ 0 ∈ C and -x il i ≤ 0 ∈ C. Therefore, we can always propagate at least one upper and one lower bound for every guarded variable x i and fix it by introducing a decision. If we cannot propagate any bound for x i , then x i is unguarded and stuck and, therefore, Slack-Intro is applicable. If we cannot fix x i by introducing a decision, then x i is unguarded and there is a conflict (see Definition 3.5.6). Guarded conflicts are resolved via the Conflict(-Div) rules. Unguarded conflicts are resolved via the unguarded conflict resolution rules. Therefore, CutSat++ has always a rule applicable unless an end state is reached.

The proof outlined above works because all unguarded conflicts encountered by CutSat++ are either the result of multiple contradicting divisibility constraints that can by combined with the rules Solve-Div-Left and Solve-Div-Right, or the conflict is expressible via a conflicting core. Since conflicting cores are only defined over constraints and propagated bounds, we have to guarantee that CutSat++ never encounters an unguarded conflict I where x i = top(I) is fixed with a decided bound. We express this property with the following invariant fulfilled by every state visited by Cut-Sat++:

Definition 3.6.7 (Eager Top-Level Propagated States). A search/conflict state S = M, C, I is called eager top-level propagated if it holds for all unguarded variables x i , all decided bounds γ = x i b i ( ∈ {≤, ≥}) in M = [[M , γ, M ]]
, and all constraints J ∈ C with top(J) = x i that: (1) all other variables contained in J are fixed in M and (2) J is no conflict in S. Proof. Let S be an eager top-level propagated state and S its successor, i.e., S =⇒ CS S. We prove this Lemma with a case distinction on the rule leading to the above transition:

1. Let the applied rule be Propagate(-Div). Then S = M , C and S = [[M , x i J b i ]], C , where ∈ {≤, ≥}. Let J ∈ C be the constraint used for propagation with b i = bound(J , x i , , M ) and J = tight(J , x i , M ) (or J = div-derive(J , x i , , M )). Then J fulfills by definition of the propagation rules the property improves(J , x i , , M ). Let the unguarded variable x j be fixed by a decided bound γ in M = [[M , γ, M ]]. Let I ∈ C be a constraint with top(I) = x j . Since S is eager top-level propagated, all variables in I are fixed in M and M . The variable x i is not fixed in M because the predicate improves(J , x i , , M ) must be true for Propagate(-Div) to be applicable. Therefore, x i is not contained in I and I is still no conflict in S. Furthermore, all variables in

I are still fixed in [[M , x i J b i ]].
We conclude that S is eager top-level propagated. C , where ∈ {≤, ≥}. We will use the eager top-level propagating strategy (Definition 3.5.6) to prove that S is an eager top-level propagated successor state. First, we consider all unguarded variables x j that are already decided in S by a decided bound γ and prove the properties for them. Since x j is already decided in S , its decided bound γ is part of M , i.e., M = [[M , γ, M ]]. Let I ∈ C be a constraint with top(I) = x j . As S is eager top-level propagated, all other variables contained in I are fixed in M and, therefore, also in M . Since L(x i , M ) < U(x i , M ) is a condition of the Decide rule, the variable x i is not fixed in M . Therefore, x i is not contained in I and I is still no conflict in S. Furthermore, all variables in

I are still fixed in [[M , x i b i ]]
. Next, we prove that S is eager top-level propagated although variable x i is newly decided. Considering Definition 3.5.6-(2a) we see that Definition 3.6.7-( 1) is fulfilled. Similarly, Definition 3.5.6-(2b) enforces Definition 3.6.7-(2). We conclude that S is eager top-level propagated.

3. Let the applied rule be Unsat(-Div) or Sat. Then the successor state S is neither a search-or conflict-state. The Lemma is thereby trivially fulfilled.

4. Let the applied rule be Forget. Then S = M , C ∪ {J} and S = M , C . Therefore, any conflict I ∈ C and any decided bound in S is also contained in S . We conclude that S is eager top-level propagated.

5. Let the applied rule be Slack-Intro. Then S = M , C , x i is stuck in S and S = M , C ∪ {-x S ≤ 0, x ix S ≤ 0, -x ix S ≤ 0} . Since x i and x S are both unguarded, we have to prove for the new constraints I ∈ {-x S ≤ 0, x ix S ≤ 0, -x ix S ≤ 0} that x k = top(I) is not decided in S/S when I is a conflict S. If the slack variable x S is decided in S/S , then Slack-Intro was already applied to another variable and -x S ≤ 0 ∈ C . Thus, -x S ≤ 0 is not a conflict in S because S is an eager top-level propagated state, which means that -x S ≤ 0 is not a conflict in S . Since x i was stuck in S , x i is also not fixed in S . Moreover, x i is the top variable in the new constraints {x ix S ≤ 0, -x ix S ≤ 0}. We conclude that S is eager top-level propagated.

6. Let the applied rule be Resolve-Weak-Cooper. Then S = M , C and S = M, C ∪R c ∪R y . Moreover, the definition of Resolve-Weak-Cooper implies that M = prefix(M , x j ) with x j = min I∈Rc {top(I)}. Therefore, M is the prefix of M without decided bounds in variables greater or equal to x j . Since x j x i for all I ∈ R c and x i = top(I), we deduce that any I ∈ R c that is a conflict has no decided bound for its top variable x i in S. Since M is a prefix of M , every conflict I ∈ C appearing in state S also appears in state S . Now it is easy to see that S is eager top-level propagated because S was eager top-level propagated.

7. Let the applied rule be Solve-Div-Right. Then S = M , C ∪{I 1 , I 2 } and S = M, C ∪ {I 1 , I 2 } . We notice that M = prefix(M , x j ) with x j = top(I 2 ). Therefore, M is the prefix of M without decided bounds in variables greater or equal to x j , which includes especially the variable x i = top(I 1 ). Thus, neither the top variable of I 1 nor the top variable I 2 is fixed by a decision. Since M is a prefix of M , every conflict I ∈ C appearing in state S also appears in state S . Now it is easy to see that S is eager top-level propagated because S was eager top-level propagated.

8. Let the applied rule be Solve-Div-Left. Then S = M , C ∪ {I 1 , I 2 } and S = M , C ∪ {I 1 , I 2 } . Since the bound sequence is the same in both states, every conflict I ∈ C appearing in state S also appears in state S . By the definition of the Solve-Div-Left rule, I 2 is no conflict in state S. Note that div-solve is an equivalence preserving transformation. Thus, if I 1 were a conflict in S and top(I 1 ) = x i fixed by a Decision, then I 1 or I 2 is a conflict in S . Therefore, I 1 is no conflict or top(I 1 ) = x i has no decided bound. Now it is easy to see that S is eager top-level propagated because S was eager top-level propagated. 9. Let the applied rule be Conflict or Conflict-Div. Then S = M , C and S = M , C , I , where I is a conflict. It is easy to see that S is eager top-level propagated because S is eager top-level propagated.

10. Let the applied rule be Resolve or Skip-Decision. Then S = [[M, γ]], C , J and S = M, C , J , where J and J are conflicts in S and S, respectively. Since M is a prefix of M , every conflict I ∈ C appearing in state S also appears in state S . Now it is easy to see that S is eager top-level propagated because S was eager top-level propagated.

11. Let the applied rule be Learn. Then S = [[M , γ]], C , I and S = M , C ∪ I, I , where I is a conflict. Since CutSat++ uses a twolayered strategy (Definition 3.5.7), I is a guarded constraint. Now it is easy to see that S is eager top-level propagated because S was eager top-level propagated.

12. Let the applied rule be Backjump. Then S = [[M , γ, M ]], C , I and S = [[M , γ ]], C , where I is a conflict in S . Since CutSat++ uses a two-layered strategy (Definition 3.5.7), I is a guarded constraint. Now it is easy to see that S is eager top-level propagated because S was eager top-level propagated.

Since the start state [[]], C 0 trivially fulfills the eager top-level propagated properties, it is clear that CutSat++ produces only eager top-level states; except for the end states. The eager top-level propagated property is so important because we will use it to show that CutSat++ resolves any conflict it encounters. In case the conflict is a guarded constraint, this is done with the CDCL based conflict rules. Otherwise, the conflict I is an unguarded constraint and CutSat++ simulates weak Cooper elimination with the unguarded conflict resolution rules. First, we use Solve-Div-Left to simulate the algorithm in Figure 3.8. This either ends with a call to Solve-Div-Right resolving the conflict or CutSat++ finds a conflicting core. Then the conflicting core is resolved with the rule Resolve-Weak-Cooper. Lemma 3.6.9 (Conflicts Progress). Let S = M, C be a state reachable by CutSat++. Let I ∈ C be a conflict in state S. Then state S is not stuck.

Proof. Assume for a contradiction that state S is stuck. W.l.o.g., we assume that x i = top(I) is the smallest variable in our order that is the top variable in a conflicting constraint I ∈ C. If x i is a guarded variable, then Conflict or Conflict-Div is applicable, which contradicts our initial assumption! Therefore, x i is an unguarded variable. Furthermore, all variables x j smaller than x i are fixed. Otherwise, we deduce for the smallest unfixed variable x j that either • x j is stuck and Slack-Intro is applicable • Propagate is applicable to a constraint I where top(I ) = x j • C contains at least two divisibility constraints I 1 , I 2 that have x j as their top variable and Solve-Div-Left or Solve-Div-Right is applicable • S contains a diophantine conflicting core (x j , I d ) and Resolve-Weak-Cooper is applicable • Decide is applicable to x j because all conditions in Definition 3.5.6- [START_REF]SMT-COMP 2018 results for QF LRA (main track)[END_REF] are fulfilled Since S is eager top-level propagated and I is a conflict with top variable x i , we know that state S contains no decided bound for x i (Definition 3.6.7 and Lemma 3.6.8). W.l.o.g., we assume that C contains at most one divisibility constraint I d with x i as its top variable. Otherwise, Solve-Div-Left or Solve-Div-Right are applicable, which contradicts our initial assumption! Let x i ≥ l i be the strictest lower bound l i = bound(x i , I l , ≥, M ) for an inequality I l ∈ C with top variable x i or l i = -∞ if there is no inequality propagating a lower bound. Let x i ≤ u i be the strictest upper bound u i = bound(x i , I u , ≤, M ) for an inequality I u ∈ C with top variable x i or u i = ∞ if there is no inequality propagating an upper bound. Since the strictly-two-layered strategy forbids the application of Forget to unguarded constraints, CutSat++ never removes an unguarded inequality. Furthermore, any bound x i b i ( ∈ {≤, ≥}) propagated from a divisibility constraint requires another bound x i b i propagated from an inequality. We deduce that

u i = ∞ if U(x i , M ) = ∞ and l i = -∞ if L(x i , M ) = -∞.
Next, we do a case distinction on whether the bounds u i and l i are finite:

1. Let u i = ∞ and l i = -∞. Then L(a i x i + p i ) = -∞ holds for all inequalities a i x i + p i ≤ 0. Thus, the conflict I is no inequality. A divisibility constraint is a conflict only if L(x i , M ) = -∞ and U(x i , M ) = ∞. This contradicts the assumption that I is a conflict.

2. Let u i = ∞ and l i ∈ Z. Then L(a i x i + p i ) = -∞ holds for all inequalities a i x i + p i ≤ 0 with a i < 0 and there exists no inequality J = a i x i + p i ≤ 0 in C with a i > 0 and top(J) = x i . Thus, the conflict I is not an inequality. A divisibility constraint is a conflict only if L(x i , M ) = -∞ and U(x i , M ) = ∞. This contradicts the assumption that I is a conflict.

3. Let l i = -∞ and u i ∈ Z. Then L(a i x i + p i ) = -∞ holds for all inequalities a i x i + p i ≤ 0 with a i > 0 and there exists no inequality J = a i x i + p i ≤ 0 in C with a i < 0 and top(J) = x i . Thus, the conflict I is not an inequality. A divisibility constraint is a conflict only if L(x i , M ) = -∞ and U(x i , M ) = ∞. This contradicts the assumption that I is a conflict.

4. Let l i , u i ∈ Z and u i < l i . Then (x i , {I l , I u }) is a conflicting core and Resolve-Weak-Cooper is applicable. This contradicts the assumption that no rule is applicable.

5. Let l i , u i ∈ Z and l i ≤ u i . Then the conflict I must be the sole divisibility constraint I d for x i . If (x i , {I l , I u , I d }) is a conflicting core, then Resolve-Weak-Cooper is applicable contradicting our initial assumption. Therefore, there exists a solution v i ∈ {l i , . . . , u i } for x i satisfying I d . Let D be the set of divisibility constraints used to propagate a bound for x i in M . All constraints D ⊆ D not contained in C, i.e., D = D \ C = D \ {I d }, were eliminated with div-solve. Since div-solve preserves equivalence, it is easy to see that there exists a set of constraints D * = D * * ∪ {I d } contained in C that implies satisfiability of D: D * = D * * ∪ {I d } → D, and D * * contains only variables x j smaller than x i . However, the set of divisibility constraints D * is fixed and satisfied under the partial assignment of M in state S. Otherwise, S would contain a conflict I ∈ D * ⊆ C with top(I ) ≺ x i , which contradicts our initial assumption! Thus, setting x i to the solution v i ∈ {l i , . . . , u i } satisfies I d in M and also D ∪ {I d }. Furthermore, all propagated constraints are satisfied if

x i is set to v i : L(x i , M ) ≤ v i ≤ U(x i , M
). This contradicts the assumption that there exists a conflict I with top(I) = x i .

The remainder of the proof follows directly the proof outline from above: Theorem 3.6.10 (CUTSAT++ Progress). Let S = M, C, J be a state reachable by CutSat++. Then S is not stuck.

Proof. Assume for a contradiction that S = M, C, J is a stuck state. If CutSat++ is a conflict state, i.e., J = , then the proof for Theorem 2 in [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] shows why standard conflict resolution cannot get stuck on a state S = M, C, J . Therefore, S = M, C, J must be a search state S = M, C . Next, we assume that all guarded variables are fixed because CutSat++ can propagate at least two bounds for every guarded variable and afterwards use decided bounds to fix them. By Lemma 3.6.9, there is no conflict in state S. Since there is no conflict, at least one variable is unfixed or rule Sat would be applicable. Therefore, there must exist a smallest unfixed and unguarded variable x i . With the Slack-Intro rules CutSat++ introduces for all variables at least one lower or upper bound. Therefore, there exists a violation to the conditions in Definition 3.5.6-(2) or Decide would be applicable to x. Since x is the smallest unfixed variable, the condition in Definition 3.5.6-(2a) holds. Definition 3.5.6-(2c) is also easy to satisfy by applications of Solve-Div-Left and Solve-Div-Right. Therefore, Definition 3.5.6-(2b) is violated. Thus, there exists a constraint

I ∈ C that is a conflict in S = [[M, γ]], C
, where γ is a decided bound for x i = top(I). By Lemma 3.6.9, it is not possible that I ∈ C is also a conflict in S or S would not be stuck. Finally, I is a conflict only in S and not S if Propagate(-Div) is applicable to I. With Solve-Div-Left and Solve-Div-Right it is relatively easy to fulfil the conditions for Definition 3.5.6-(1) and, therefore, Propagate(-Div) is applicable. We conclude that CutSat++ has always one applicable rule, which is a contradiction to our assumption!

Completeness

All CutSat++ rules are sound, i.e., if M i , C i , I i =⇒ CS M j , C j , I j , then any satisfiable assignment ν for C j is also a satisfiable assignment for C i . The rule Resolve-Weak-Cooper is sound because of the Lemmas 3.4.4 and 3.4.5. The soundness of Solve-Div-Left and Solve-Div-Right follows from the fact that div-solve is an equivalence preserving transformation. The soundness proofs for all other rules are either trivial or given by [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF].

Summarizing, CutSat++ is terminating, sound, and never reaches a stuck state. In combination with the fact that Sat is applicable only if a satisfiable solution ν[M ] is found and that Unsat and Unsat-Div detect trivially unsatisfiable constraints, these facts imply completeness: Theorem 3.6.11 (CUTSAT++ Completeness). If CutSat++ starts from a start state [[]], C 0 , then it either terminates in the unsat state and C 0 is unsatisfiable, or it terminates with ν, sat where ν is a satisfiable assignment for C 0 .

Proof. By Theorem 3.6.6, CutSat++ is terminating. By [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] and the Lemmas 3.2.1, 3.4.4, and 3.4.5, CutSat++ is sound. By Theorem 3.6.10, Cut-Sat++ never reaches a stuck state. Since CutSat++ is terminating and never reaches a stuck state, every application of CutSat++ ends via the rules Sat, Unsat, or Unsat-Div in one of the end states. The rule Sat is only applicable in a state M, C where ν[M ] satisfies C and because of soundness also C 0 . The rules Unsat and Unsat-Div are only applicable to states M, C, I where the constraint set C contains a trivially unsatisfiable constraint. When CutSat++ encounters a trivially unsatisfiable constraint, then the soundness of the CutSat++ rules guarantees that C 0 is unsatisfiable.

Summary

The starting point of our work was an implementation of the CutSat [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF] calculus as a theory solver for hierarchic superposition [START_REF] Fietzke | Superposition as a decision procedure for timed automata[END_REF]. In that course, we observed divergence for some of our problems. The analysis of those divergences led to the development of the CutSat++ calculus presented in this chapter, which is, as far as we know, the first sound, complete, and terminating calculus for linear integer problems based on the model assumption and conflict learning approach motivated by CDCL style SAT solving.

CutSat++ efficiently handles problems over guarded variables, i.e., variables with a constant upper and lower bound. On problems with unguarded variables (e.g. unbounded problems) the CDCL style calculus alone is not guaranteed to terminate. Hence, we combine it with a lazy quantifier elimination procedure (called unguarded conflict resolution) that transforms a problem containing unguarded variables into one where feasibility depends only on guarded ones. The quantifier elimination procedure is called lazy because we only apply it to so-called conflicting cores, which are canonized forms of the unguarded conflicts encountered by the CDCL style algorithm. This allows us to avoid certain cases of worst-case exponential behavior that we would otherwise observe by using a quantifier elimination procedure alone.

123

Chapter 4

Fast Cube Tests (for Linear Arithmetic Constraint Solving)

Finding a mixed/integer solution for a polyhedron that is defined by a system of linear inequalities Ax ≤ b is a well-known NP-complete problem [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]. Systems of linear inequalities have many real-world applications so that this problem has been investigated in different research areas, e.g., in optimization via (mixed) integer linear programming (MILP) [START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Kannan | On the computational complexity of integer programming problems[END_REF][START_REF] Mitchell | Branch-and-cut algorithms for combinatorial optimization problems[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF] and in constraint solving via satisfiability modulo theories (SMT) [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF][START_REF] Bromberger | Linear integer arithmetic revisited[END_REF][START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF][START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF].

For commercial MILP implementations, it is standard to integrate preprocessing techniques, heuristics, and specialized tests [START_REF] Glover | General purpose heuristics for integer programming -part I[END_REF][START_REF] Glover | General purpose heuristics for integer programming-part II[END_REF][START_REF] Hillier | Efficient heuristic procedures for integer linear programming with an interior[END_REF][START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Martin | Preprocessing and probing techniques for mixed integer programming problems[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF][START_REF] Wallace | ZI round, a MIP rounding heuristic[END_REF]. Although these techniques are not complete, they are much more efficient on their designated target systems of linear inequalities than a complete algorithm alone. There actually exist specialized techniques for many classes of real-world problems representable as systems of linear inequalities. Therefore, commercial MILP solvers are efficient on many real-world inputs-even though the problem is NP-complete, in general.

The SMT community is still in the process of developing their own variety of specialized tests. It is even a big challenge to adopt the tests from the MILP community so that they still fit the requirements of SMT solving. One of those requirements is that SMT theory solvers have to solve a large number of incrementally connected, small systems of linear inequalities efficiently. Therefore, exploiting the incremental connection is key for making SMT theory solvers efficient [START_REF] Faure | SAT modulo the theory of linear arithmetic: Exact, inexact and commercial solvers[END_REF]. In contrast, MILP solvers typically target one large system. The same holds for their specialized tests, which are not well suited to exploit incremental connections.

125

In this chapter, we present two tests tailored toward SMT solvers: the largest cube test and the unit cube test. The idea is to find hypercubes that are contained inside the input polyhedron and guarantee the existence of a mixed/integer solution. Due to computational complexity, we will restrict ourselves to only those hypercubes that are parallel to the coordinate axes. The largest cube test finds a hypercube with maximum edge length contained in the input polyhedron, determines its rational valued center, and rounds it to a potential mixed/integer solution. The unit cube test determines if a polyhedron contains a hypercube with edge length one, which is the minimal edge length that guarantees a mixed/integer solution.

Most linear arithmetic theory solvers for SMT are based on a branchand-bound algorithm on top of a simplex algorithm (see Chapter 2.7). They search for a solution at the surface of a polyhedron. However, our tests search in the interior of the polyhedron. This gives them an advantage on polyhedra with a large number of integer solutions, e.g., absolutely unbounded inequality systems (see Chapter 2.8).

SMT theory solvers are designed to efficiently exchange bounds (see Chapter 2.7 and [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]). This efficient exchange is the main reason why SMT theory solvers exploit the incremental connection between the different inequality systems so well. Our unit cube test also requires only an exchange of bounds. After applying the test, we can easily recover the original system by reverting to the original bounds. In doing so, the unit cube test conserves the incremental connection to the different original systems. We make a similar observation about the largest cube test.

Our contributions are as follows: we define the linear cube transformation (Corollary 4.2.2) that allows us to efficiently compute whether a system of inequalities Ax ≤ b contains a hypercube of edge length e in Section 4.2. The most remarkable fact about this transformation is that it solely changes the bounds b of the inequalities. Based on this transformation, we develop in Section 4.3 two tests: the largest cube test and the unit cube test. For absolutely unbounded inequality systems, both tests always succeed (Lemma 4.4.1). Inside the SMT-LIB benchmarks [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF], there are almost one thousand absolutely unbounded problem instances, and we show the advantage of our cube tests on these instances by comparing our implementation of the cube test with several state-of-the-art SMT solvers in Section 4.5. Our implementation is not only several orders of magnitudes faster, but it also solves all instances, which most SMT solvers do not (Figure 4.8).

Related Work and Preliminaries

This chapter is based on two publications with Christoph Weidenbach as coauthor [START_REF] Bromberger | Fast cube tests for LIA constraint solving[END_REF][START_REF] Bromberger | New techniques for linear arithmetic: cubes and equalities[END_REF] and the techniques in this chapter focus on the interaction of geometric objects. In the case of a system of inequalities, this means the polyhedron defined by Ax ≤ b. This is the reason why we use in this chapter polyhedron as an alternative name for systems of inequalities Ax ≤ b. The other geometric objects we are considering are cubes and flat cubes. Their definitions can be found in Chapter 2.9.

This chapter focuses on all three theories of linear arithmetic. Our tests themselves are formulated as problems in the theory of linear rational arithmetic. However, the goal of our tests is to find mixed/integer solutions for problems in the theory of linear mixed/integer arithmetic. We first present all of our techniques for linear integer arithmetic. In Section 4.3.3, we then extend them to linear mixed arithmetic. To avoid confusion between the various theories, we always specify the type of solution/satisfiability.

The constraints in this chapter are non-strict inequalities and they are either formatted according to the vector representation, i.e., a T i x ≤ b i (see also Chapter 2.2.1), or the standard representation, a i1 x 1 + . . . + a in x n ≤ b i (see also Chapter 2.2.1). Other constraints have to be reduced to non-strict inequalities with the techniques presented in Chapter 2.3.

This chapter builds on the basics of linear algebra (Chapter 2.1) and linear arithmetic (Chapter 2.2), on the concept of implied constraints (Chapter 2.5), and on the definitions of (un)bounded and (un)guarded problems and variables (Chapter 2.8). Knowledge of standard arithmetic decision procedures for SMT solvers (Chapter 2.7) is not required to understand the tests that we propose here, but it is necessary to fully understand the practical usefulness of the tests with regard to SMT solving.

There also exist several publications by other authors that are highly relevant to the contributions presented in this chapter. The first one is by Hillier [START_REF] Hillier | Efficient heuristic procedures for integer linear programming with an interior[END_REF], who was aware of the unit cube test, but applied it only to cones (a special class of polyhedra) as a subroutine of the heuristic he presents in the same paper. His work never mentioned applications beyond cones, nor did he prove any structural properties connected to hypercubes. Hillier's heuristic tailored for MILP optimization lost popularity as soon as interior point methods became efficient in practice [START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF]. Nonetheless, our cube tests remain relevant for SMT theory solvers because there are no competitive incremental interior point methods known.

The second related work is by Bobot et al. [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF]. They discuss in their paper [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF] relations between hypercubes and polyhedra including absolute unboundedness and positive linear combinations between inequalities. Our largest cube test can also detect these relations because it is, with some minor changes, the dual of the linear optimization problem of Bobot et al.

In contrast to the linear optimization problem of Bobot et al., our tests are closer to the original polyhedron and, therefore, easier to construct. Our cube tests also produce sample points and find solutions for polyhedra that are bounded.

Another method that provides a sufficient condition for the existence of an integer solution is the dark shadow of the Omega Test [START_REF] Pugh | The omega test: a fast and practical integer programming algorithm for dependence analysis[END_REF]. The dark shadow is based on Fourier-Motzkin elimination and its worst-case runtime is double exponential. Although not practically advantageous, formulating the unit cube test through Fourier-Motzkin elimination allows us to put the sufficient conditions of the two methods in relation. Fourier-Motzkin elimination eliminates the variable x j from a problem by combining each pair of inequalities a j x j + p j ≤ 0 and -b j x j + q j ≤ 0 (with a j , b j > 0) into a new inequality a j q j + b j p j ≤ 0. The dark shadow creates a stronger version (a j q j + b j p j ≤ a j + b ja j b j ) of the combined inequality to guarantee the existence of an integer solution for x j . Formulating the unit cube test through Fourier-Motzkin elimination makes the combined inequality even stronger (a j q j + b j p j ≤ -a j b j ). This means that the sufficient condition of the dark shadow subsumes the condition of the unit cube test. Still, our unit cube test is definable as a linear program and it is, therefore, computable in polynomial time. So the better condition of the dark shadow comes at the cost of being much harder (doubly exponential) to compute.

Fitting Cubes into Polyhedra

We say that a cube C e (z) (see Chapter 2.9.2 for the formal definition) fits into a polyhedron defined by Ax ≤ b if all points inside the cube C e (z) are solutions of Ax ≤ b, or formally: C e (z) ⊆ Q δ (Ax ≤ b). In order to compute this, we transform the polyhedron Ax ≤ b into another polyhedron Ax ≤ b . For this new polyhedron, we merely have to test whether the cube's center point z is a solution (z ∈ Q δ (Ax ≤ b )) in order to also determine whether the cube C e (z) fits into the original polyhedron. This is a simple test that requires only evaluation. We call this entire transformation the linear cube transformation.

We start explaining the linear cube transformation by looking at the case where the polyhedron is defined by a single inequality a T i x ≤ b i . A cube C e (z) fits into the inequality a T i x ≤ b i if all points inside the cube C e (z) are solutions of a T i x ≤ b i , or formally: ∀x ∈ C e (z). a T i x ≤ b i . We can think of a T i x as an objective function that we want to maximize and see b i as a guard for the maximum objective of any solution in the cube. Thus, we can express the universal quantifier in the above equation as an optimization problem (see Figure 4.1): max{a T i x : x ∈ C e (z)} ≤ b i . This also means that all points in x ∈ C e (z) satisfy the inequality a T i x ≤ b i if a x ∈ C e (z)} for the objective function a T i x satisfies the inequality a T i x * ≤ b i . We can formalize the above optimization problem as a linear program: maximize a T i x subject to z j -e 2 ≤ x j ≤ z j + e 2 for j = 1, . . . , n . However, for the case of cubes, there is an even easier way to determine the maximum objective value. Since every cube is a bounded polyhedron, one of the points with maximum objective value is a vertex v ∈ C e (z). A vertex v of the cube C e (z) is one of the points with maximum distance to the center z (see Figure 4.2), or formally: v = z 1 ± e 2 , . . . , z n ± e

2

T . If we insert the above equation into the objective function a T i x, we get:

a T i z 1 ± e 2 , . . . , z n ± e 2 T = a T i z + e 2 n
j=1 ±a ij , which in turn is maximal if we choose v such that ±a ij is always positive:

a T i v = a T i z + e 2 n j=1 |a ij | = a T i z + e 2 a i 1 . Hence, we transform the inequality a T i x ≤ b i into a T i x ≤ b i -e 2 a i 1 , and C e (z) fits into a T i x ≤ b i if a T i z ≤ b i -e 2 a i 1 . Lemma 4.2.

(Linear Cube Transformation).

Let C e (z) be a cube and

a T i x ≤ b i be an inequality. All x ∈ C e (z) fulfill a T i x ≤ b i if and only if a T i z ≤ b i -e 2 a i 1 . Proof. ⇒: Assume that all x ∈ C e (z) fulfill a T i x ≤ b i . ( 4
a T i v = (4.2) e 2 a T i (sgn(a i1 ), . . . , sgn(a in )) T + a T i z = e 2 ( n j=1 a ij • sgn(a ij )) + a T i z = e 2 ( n j=1 |a ij |) + a T i z . 129
⇐: Assume that e 2 n j=1 |a ij | + a T i z ≤ b i . (4.
3) By the definition of the cube C e (z), it follows that for all points x ∈ C e (z), there exists a vector d ∈ R n such that d j ∈ [-1, 1] and

z := x -e 2 d. (4.4) For d j ∈ [-1, 1], it follows that |a ij | -a ij d j ≥ 0 . (4.5) Hence, b i ≥ (4.3) e 2 n j=1 |a ij | + a T i z = (4.4) e 2 n j=1 |a ij | + a T i x -e 2 a T i d = e 2 n j=1 (|a ij | -a ij d j ) + a T i x ≥ (4.5) a T i x .
Next, we look at the case where multiple inequalities Until now, we have discussed how to use the linear cube transformation to determine if one cube C e (z) with fixed center point z fits into a polyhedron. A generalization of this problem determines whether a polyhedron Ax ≤ b contains a cube of edge length e at all. Actually, a closer look at the transformed polyhedron Ax ≤ b reveals that the linear cube transformation (b i = b i -e 2 a i 1 ) is dependent only on the edge length e of the cube. Therefore, the solutions Q δ (Ax ≤ b ) of the transformed polyhedron Ax ≤ b are exactly all center points of cubes with edge length e that fit into the original polyhedron Ax ≤ b (see Figure 4.3). By determining the rational satisfiability of the transformed polyhedron Ax ≤ b , we can now also determine whether a polyhedron Ax ≤ b contains a cube of edge length e at all. If we choose a suitable algorithm, e.g., the simplex algorithm, then we even get the center point z of a cube C e (z) that fits into Ax ≤ b. This observation is the foundation for the cube tests that we present in Section 4.3.

a T i x ≤ b i (for i = 1, . . . , m) define the polyhedron Ax ≤ b. Since Q δ (Ax ≤ b) is the intersection of all Q δ (a T i x ≤ b i ),

Fast Cube Tests

A polyhedron Ax ≤ b has an integer solution if and only if

Z(Ax ≤ b) = Q δ (Ax ≤ b) ∩ Z n = ∅, i.e.
, if the set of rational solutions contains an integer point. In this section, we show how to use the linear cube transformation to find such an integer solution. In contrast to arbitrary polyhedra, determining whether a cube C e (z) contains an integer point is easy. Because of the cubes symmetry, it is enough to test whether it contains a closest integer point z to the center z (see also Corollary 2.9.2).

Note that every point z ∈ Q n δ is also a cube C 0 (z) of edge length 0. In order to be efficient, our tests look only at cubes with special properties. In the case of the largest cube test, we check for an integer solution in one of the largest cubes fitting into the polyhedron Ax ≤ b. In the case of the unit cube test, we look for a cube of edge length one, which always guarantees an integer solution. Due to these restrictions, both tests are not complete but very fast to compute.

Largest Cube Test

A well-known test, implemented in most ILP solvers, is simple rounding (see also Chapter 2.1.2 & 2.7.4). For simple rounding, the ILP solver computes a rational solution x for a set of inequalities, rounds it to a closest integer x , and determines whether this point is an integer solution. Not all types of rational solutions are good candidates for this test to be successful. Especially surface points, such as vertices, are not good candidates for roun-ding. Vertices are, however, the usual output of the simplex algorithm. For many polyhedra, center and interior points z are a better choice because all integer points adjacent to z are solutions, including a closest integer point z .

We now use the linear cube transformation (Section 4.2) to calculate a rational center point with the simplex algorithm. The center point we calculate is the center point of a largest cube that fits into the polyhedron Ax ≤ b (see Figure 4.4). We determine the center z of this largest cube and the associated edge length e with the following linear program (LP):

maximize x e subject to Ax + a xe 2 ≤ b, where a i = a i 1 x e ≥ 0 . This linear program employs the linear cube transformation from Section 4.2. The only generalization is a variable x e for the edge length instead of a constant value e. Additionally, this linear program maximizes the edge length as an optimization goal. If the resulting maximum edge length is unbounded, the original polyhedron contains cubes of arbitrary edge length (see Figure 4.5) and, thus, infinitely many integer solutions. Since the linear program contains all rational solutions of the original polyhedron (see x e = 0), the original polyhedron is empty if and only if the above linear program is rationally unsatisfiable. If the maximum edge length is a finite value e, we use the resulting assignment z for the variables x as a center point and C e (z) is a largest cube that fits into the polyhedron. From the center point, we round to a closest integer point z and determine if it fits into the original polyhedron. If this is the case, we are done because we have found an integer solution for Ax ≤ b. Otherwise, the largest cube test does not know whether or not Ax ≤ b has an integer solution. An example for the latter case, are the following inequalities: -2x 1 -2x 2 ≤ -3, 4x 1x 2 ≤ 1, and -3x 1 + 2x 2 ≤ 3. These inequalities have exactly one integer solution (1, 3) T , but the largest cube contained by the inequalities has edge length e = 5 12 and center point ( 3 8 , 37 24 ) T , which rounds to (0, 2) T (see Figure 4.6). The largest cube test also upholds the incremental advantages of Dutertre and de Moura's version of the dual simplex algorithm (see also Chapter 2.7.1 and [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]). The only difference is the extra column a xe 2 , which the theory solver can internally create while it is notified of all potential arithmetic literals. Adding this column from the start does not influence the correctness of the solution because x e ≥ 0 guarantees that the largest cube test is rationally satisfiable exactly when the original inequalities Ax ≤ b are rationally satisfiable. Even for explanations of rational unsatisfiability, it suffices to remove the bound x e ≥ 0 to obtain an explanation for the original inequalities Ax ≤ b. The only disadvantage is the additional variable x e , which only shrinks the search space when it is increased. Therefore, increasing x e can never resolve any conflicts during the satisfiability search. The

Unit Cube Test

Most SMT solvers [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF] implement a simplex algorithm (see also Chapter 2.7.1 and [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]) that is specialized towards satisfiability and not towards optimization. Therefore, a test based on optimization, such as the largest cube test, does not fit well with existing implementations. As an alternative, we have developed a second test based on cubes that does not need optimization.

We avoid optimization by fixing the edge length e to the value 1 for all the cubes C e (z) we consider (see Figure 4.7). We do so because cubes C 1 (z) of edge length 1 are the smallest cubes to always guarantee an integer solution independent of the center point z. A cube with edge length 1 is also called a unit cube. To prove this guarantee, we first fix e = 1 in the definition of cubes,

C 1 (z) = x ∈ Q n δ : ∀j ∈ 1, . . . , n. |x j -z j | ≤ 1 2
, and look at the following property for the rounding operator . : ∀z j ∈ Q δ .| z jz j | ≤ 1 2 . We see that any unit cube contains a closest integer z to its center point z. Furthermore, 1 is the smallest edge length that guarantees an integer solution for a cube with center point z = (. . . , 1 2 , . . .) T . Thus, 1 is the smallest value that we can fix as an edge length to guarantee an integer solution for all cubes C 1 (z).

Our second test tries to find a unit cube that fits into the polyhedron Ax ≤ b and, thereby, a guarantee for an integer solution for Ax ≤ b. Again, we employ the linear cube transformation from Section 4.2 and obtain the linear program:

Az ≤ b , where b i = b i -1 2 a i 1 . In addition to being a linear program without an optimization objective, we only have to change the row bounds b i of the original inequalities. In Dutertre and de Moura's version of the dual simplex algorithm (see also Chapter 2.7.1 and [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]), which is implemented in many SMT solvers [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF], such a change of bounds is already part of the framework so that integrating the unit cube test into theory solvers is possible with only minor adjustments to the existing implementation. Since our unit cube test requires only an exchange of bounds, we can easily return to the original polyhedron by reverting the bounds. In doing so, the unit cube test upholds the incremental connection between the different original polyhedra.

Cube Tests for Linear Mixed Arithmetic

We can also extend our cube tests to the theory of linear mixed arithmetic. As mentioned in Chapter 2.2, our variables x = (x 1 , . . . , x n ) T are actually partitioned into two vectors: the rational variables (x 1 , . . . , x n 1 ) T and the integer variables (x n 1 +1 , . . . , x n 1 +n 2 ) T with n = n 1 + n 2 . Based on this partitioning, we also split the coefficient matrix A into two matrices A = (R, S), where R = (r 1 , . . . , r m ) T ∈ Q m×n 1 defines the coefficients for the rational variables and S = (s 1 , . . . , s m ) T ∈ Q m×n2 defines the coefficients for the integer variables.

Because only integer variables need to be assigned to integer values, tests like simple rounding should be restricted to integer variables. For instance, if z is a rational solution for the overall polyhedron, then simple rounding applies . only to the components of z that correspond to integer variables. 1 The same holds for our fast cube tests. Instead of looking for hypercubes of the same dimension n as the number of total variables, we are looking for hypercubes of dimension n 2 that expand in the directions that correspond to integer variables, but are flat in the directions that correspond to rational variables. Such a hypercube of dimension n 2 with center point z and edge length e is a flat cube F e (z) (see Chapter 2.9.3 for the formal definition).

We can also modify the linear cube transformation so that we can compute whether a polyhedron Ax ≤ b contains a flat cube F e (z) that is less than full dimensional: Lemma 4.3.1 (Flat Cube Transformation). Let F e (z) be a flat cube and Ax ≤ b be a polyhedron with A = (R, S), R = (r 1 , . . . , r m ) T ∈ Q m×n 1 , and

S = (s 1 , . . . , s m ) T ∈ Q m×n 2 . F e (z) ⊆ Q δ (Ax ≤ b) if and only if Az ≤ b , where b i = b i -e
2 s i 1 . Proof. Works the same as the proofs in Section 4.2.

Since the hypercube F e (z) only expands in the directions that correspond to integer variables, the inequality bounds b of the modified linear cube transformation are only influenced by the coefficients of the integer variables. Using Lemma 4.3.1, we can now modify our fast cube tests so that they work for linear mixed arithmetic. For the largest cube test, we compute the center point of a largest flat cube F e (z) that fits into the polyhedron Ax ≤ b. We determine the center z of this largest flat cube and the associated edge length e with the following LP: maximize x e subject to Ax + s xe 2 ≤ b, where s i = s i 1 x e ≥ 0 . From the resulting center point z we receive a candidate mixed solution by applying the rounding operator . to the components of z that correspond to integer variables. For the unit cube test, we search for a cube F 1 (z) that is flat in the directions that correspond to rational variables, has edge length 1, and fits into the polyhedron Ax ≤ b. A linear program that accomplishes this task is: Ax ≤ b , where b i = b i -1 2 s i 1 . Again, 1 is the smallest value that we can fix as an edge length to guarantee a mixed solution for all cubes F 1 (z).

Absolutely Unbounded Polyhedra

While our tests are useful for many types of polyhedra, the motivation for our tests stems from a special type of polyhedron, a so-called absolutely unbounded polyhedron or also called infinite lattice width polyhedron (see Chapter 2.8 and [START_REF] Kannan | Covering minima and lattice point free convex bodies[END_REF]). A polyhedron Ax ≤ b is absolutely unbounded if for every objective h ∈ Q n \ {0 n }, either its maximum or minimum objective value is unbounded:

∀h ∈ Q n \ {0 n }. sup h T x : x ∈ Q δ (Ax ≤ b) = ∞ or inf h T x : x ∈ Q δ (Ax ≤ b) = -∞ .
Absolutely unbounded polyhedra seem trivial at first glance because their interior expands arbitrarily far in all directions (see Figure 4.5). Therefore, an absolutely unbounded polyhedron contains an infinite number of integer solutions [START_REF] Kannan | Covering minima and lattice point free convex bodies[END_REF]. Nonetheless, many SMT solvers are inefficient on those polyhedra because they use a branch-and-bound approach with an underlying simplex solver (see Chapter 2.7 or [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]). Although such an approach terminates inside finite a priori bounds see Chapter 2.7 and [START_REF] Papadimitriou | On the complexity of integer programming[END_REF]), it does not explore the infinite interior, but rather directs the search along the rational solutions suggested by the simplex solver: the vertices of the polyhedron. Thus, the SMT solvers concentrate their search on a bounded part of the polyhedron. This bounded part contains only a finite number of integer solutions, whereas the complete interior contains infinitely many integer solutions. The advantage of our cube tests is that they actually exploit the infinite interior because absolutely unbounded polyhedra contain cubes for every edge length (see Figure 4.5). Our tests are, therefore, always successful on absolutely unbounded polyhedra and usually need only a small number of pivoting steps before finding a solution.

Lemma 4.4.1 (Unbounded Cubes Lemma). Let Ax ≤ b be a polyhedron. Let a ∈ Z m be a vector such that its components are a i = a i 1 . Then the following two statements are equivalent:

(1) Ax ≤ b contains a cube C e (z) for every e ≥ 0, and (2) Ax ≤ b is absolutely unbounded. Or formally:

(1) ∀e ∈ Q n δ .(e ≥ 0) → (∃z ∈ Q n δ . C e (z) ⊆ Q δ (Ax ≤ b)) , (2) ∀h ∈ Q n \ {0 n }.sup h T x : x ∈ Q δ (Ax ≤ b) = ∞ or inf h T x : x ∈ Q δ (Ax ≤ b) = -∞ .
Proof. (1) ⇒ (2): We first assume that Ax ≤ b contains a cube C e (z) for every e ≥ 0. Note that the center point z depends on the edge length e. Furthermore, we define the function: width(h, S) = sup h T x : x ∈ S) + sup -h T x : x ∈ S) (4.6) for every vector h ∈ Q n \ {0 n } and for every set of points S ⊆ Q n δ . Then we prove that: lim e→∞ width(h, C e (.)) → ∞ . In Section 4.2, we have shown that:

sup h T x : x ∈ C e (z) = h T z + e 2 • h 1 , and (4.7) sup -h T x : x ∈ C e (z) = -h T z + e 2 • h 1 . ( 4
(Ax ≤ b)) = ∞. Since Q δ (Ax ≤ b) is also convex, it must hold that: sup h T x : x ∈ Q δ (Ax ≤ b) = ∞ or inf h T x : x ∈ Q δ (Ax ≤ b) = -∞ .
(2) ⇒ (1): By contradiction. Assume that Ax ≤ b is absolutely unbounded but that there exists an e ∈ Q δ (with e ≥ 0) such that Ax ≤ b contains no cube C e (z) of edge length e. By Corollary 4.2.2, Ax ≤ b contains no cube C e (z) of edge length e implies that Ax ≤ b -e 2 • a is rationally unsatisfiable. By Farkas' Lemma, Ax ≤ b -e 2 • a is rationally unsatisfiable implies that there exists a y ∈ Q m such that: (a) y i ≥ 0 for all i ∈ {1, . . . , m}, (b) y k > 0 for at least one k ∈ {1, . . . , m}, (c) y T A = (0 n ) T , and (d) 0 > y T b -e 2 • y T a .

Because of (b), we can transform the equality (c) into the following form:

a T k = -m i=1,i =k y i y k a T i .
(4.9) By multiplying (4.9) with an x ∈ Q δ (Ax ≤ b), we get:

a T k x = -m i=1,i =k y i y k a T i x . Since a T i
x ≤ b i and y i ≥ 0, we get a finite lower bound for a T k x:

a T k x = -m i=1,i =k y i y k a T i x ≥ -m i=1,i =k y i y k b i . Thus, the upper bound sup a T k x : x ∈ Q δ (Ax ≤ b) ≤ b k < ∞ and the lower bound inf a T k x : x ∈ Q δ (Ax ≤ b) ≥ -m i=1,i =k y i y k b i > -∞
are finite, which contradicts the assumption that Ax ≤ b is absolutely unbounded.

Experiments

We have found instances of absolutely unbounded polyhedra in some classes of the SMT-LIB benchmarks [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF]. These instances are 229 of the 233 dillig benchmarks designed by Dillig et al. [START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF], 503 of the 591 CAV-2009 benchmarks also by Dillig et al. [START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF], 229 of the 233 slacks benchmarks which are the dillig benchmarks extended with slack variables [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], and 19 of the 37 prime-cone benchmarks, that is, "a group of crafted benchmarks encoding a tight n-dimensional cone around the point whose coordinates are the first n prime numbers" [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. The remaining problems (4 from dillig, 88 from CAV-2009, 4 from slacks, and 18 from prime-cone) are not absolutely unbounded because they are either tightly bounded or integer unsatisfiable. For our experiments, we look only at the instances of those benchmark classes that are actually absolutely unbounded.

Using these benchmark instances, we have confirmed our theoretical assumptions (Lemma 4.4.1) in practice. We integrated the unit cube test into our own branch-and-bound solver SPASS-IQ (v0.3)2 and ran it on the absolutely unbounded instances; once with the unit cube test turned on (SPASS-IQ+uc) and once with the test turned off (SPASS-IQ). For every problem, SPASS-IQ+uc applies the unit cube test exactly once. This application happens before we start the branch-and-bound approach. To evaluate the efficiency of the unit cube test, we compared SPASS-IQ to several other solvers for systems of linear inequalities. [START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF], Yices (v2.6.0) [START_REF] Dutertre | Yices 2.2[END_REF], and Z3 (v4.8.1) [START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF]. All these solvers employ a branch-and-bound approach with an underlying dual simplex solver [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]. The only exception are MathSAT5 and Z3, which, subsequent to our first publication on the unit cube test [START_REF] Bromberger | Fast cube tests for LIA constraint solving[END_REF], now also perform the unit cube test in advance. That is why we also test MathSAT5 and Z3 once with the unit cube test turned on (MathSAT5+uc and Z3+uc) and once with the test turned off (MathSAT5 and Z3).

The solvers had to solve each problem in under 40 minutes. For the experiments, we used a Debian Linux cluster and allotted to each problem and solver combination 2 cores of an Intel Xeon E5620 (2.4 GHz) processor, 4 GB RAM, and 40 minutes. Figure 4.8 lists the results of the different solvers (column one) on the different benchmark classes (row one). Row two lists the number of benchmark instances we considered for our experiments. For each combination of benchmark class and solver, we have listed the number of instances the solver could solve in the given time as well as the total time (in seconds) of the instances solved (columns labelled with "solved" and "time", respectively).

Our solver that employs the unit cube test solves all instances with the application of the unit cube test and is 25 times faster than our solver without the test. The SMT theory solvers in their standard setting were not able to solve all instances within the allotted time. Moreover, our unit cube test was over 100 times faster than any state-of-the-art SMT solver without the unit cube test. The results for MathSAT5 and Z3 further support the superiority of the test.

Comparison with Ctrl-Ergo

Second, we compared our unit cube test with the Ctrl-Ergo solver, which includes a subroutine that is essentially the dual to our largest cube test [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF]. As expected, both approaches are comparable for absolutely unbounded polyhedra. In order to also compare the two approaches on benchmarks that are not absolutely unbounded, we created the rotate benchmarks by adding the same four inequalities to all absolutely unbounded instances of the dillig benchmarks. These four inequalities essentially describe a square bounding the variables x 0 and x 1 in an interval [-u, u]. For a large enough choice of u (e.g., u = 2 10 ), the square is so large that the benchmarks are still integer satisfiable and not absolutely trivial for branch-and-bound solvers. To add a challenge, we rotated the square by a small factor 1/r, which resulted in the following four inequalities:

-b • r • r + r ≤ b • r • x 0 -x 1 ≤ b • r • r -r , and -b • r • r + r ≤ x 0 + b • r • x 1 ≤ b • r • r -r .
These changes have nearly no influence on SPASS-IQ, and two SMT solvers even benefit from the proposed changes. For Ctrl-Ergo the rotate benchmarks are very hard because its subroutine detects only absolutely unbounded polyhedra. If the polyhedron is not absolutely unbounded, then Ctrl-Ergo starts its search from the boundaries of the polyhedron instead of looking at the polyhedron's interior. We can even control the number of iterations (r 2 ) Ctrl-Ergo spends on the parts of the boundary without any integer solutions if we choose r accordingly (e.g., r = 2 10 ). In contrast, we use our cube tests to also extract interior points for rounding. This difference makes our tests much more stable under small changes to the polyhedron.

Most problems in the linear integer arithmetic SMT-LIB benchmarks with finite lattice width (i.e., that are not absolutely unbounded) can be solved without using any actual integer arithmetic techniques. A standard simplex solver for the rationals typically finds a rational solution for such a problem that is also an integer solution. Applying the unit cube test on these trivial problem classes is a waste of time. In the worst case, it doubles the eventual solution time. For these examples it is beneficial to first compute a general rational solution and to check it for integer satisfiability before applying the unit cube test. This has the additional benefit that rational unsatisfiable problems are filtered out before applying the unit cube test. The unit cube test is also guaranteed to fail on problems containing boolean variables, i.e., variables that are either 0 or 1, unless they are absolutely trivial and describe a unit cube themselves. Whenever the problem contains a boolean variable, it is beneficial to skip the unit cube test. This is also the reason why we provide no experimental results for the theory of linear mixed arithmetic, i.e., the few mixed benchmarks available in the SMT-LIB all contain boolean variables.

Comparison with MILP Solvers

Third, we compared our unit cube test with several solvers for mixedinteger programming (MILP) (see Figure 4.9): the two non-commercial solvers GLPK (v4.65) [START_REF] Makhorin | GNU Linear Programming Kit (GLPK)[END_REF] and SCIP (v6.0.0) [START_REF] Gleixner | The SCIP Optimization Suite 6.0[END_REF] as well as the commercial solver Gurobi (v7.52) [START_REF]Gurobi Optimizer Reference Manual[END_REF]. For these experiments, we used the same benchmarks-although converted into the MPS (Mathematical Programming System) format-and the same experiment parameters as for our experiments with the SMT solvers. In General, mixed-integer programming solvers have an advantage over standard SMT theory solvers because (i) they are not required to be exact and sound, which allows them to use floatingpoint arithmetic, and (ii) they are not required to be incrementally efficient, which means they can use much more elaborate techniques. Despite these advantages, SPASS-IQ is faster and solves more problems from the absolutely unbounded benchmarks than GLPK and SCIP. The reason is that GLPK and SCIP rely-like the state-of-the-art SMT theory solvers-on a branch-and-bound approach with an underlying simplex solver, which means they also focus their search on the vertices of the polyhedron instead of the polyhedron's interior. Gurobi, on the other hand, is faster than SPASS-IQ on the absolutely unbounded benchmarks because (i) it uses an interior point method [START_REF] Karmarkar | A new polynomial-time algorithm for linear programming[END_REF] to compute the first rational solution for its branch-andbound approach and only then switches to its simplex solver and (ii) it uses floating-point arithmetic, which is more efficient than exact arithmetic.

The experiments with Gurobi give the impression that interior point methods are an efficient alternative to our unit cube test. At least for now, this impression is only correct for mixed-integer programming and not for SMT theory solvers. Interior point methods perform worse in the context of SMT theory solvers because the currently competitive interior point methods are not incrementally efficient, which is one of the most important properties for an efficient SMT theory solver [START_REF] Faure | SAT modulo the theory of linear arithmetic: Exact, inexact and commercial solvers[END_REF].

On Other SMT-LIB Benchmarks

Most problems in the linear integer arithmetic SMT-LIB benchmarks with finite lattice width (i.e., that are not absolutely unbounded) can be solved without using any actual integer arithmetic techniques. A standard simplex solver for the rationals typically finds a rational solution for such a problem that is also an integer solution. Applying the unit cube test on these trivial problem classes is a waste of time. In the worst case, it doubles the eventual solution time. For these examples it is beneficial to first compute a general rational solution and to check it for integer satisfiability before applying the unit cube test. This has the additional benefit that rational unsatisfiable problems are filtered out before applying the unit cube test. The unit cube test is also guaranteed to fail on problems containing boolean variables, i.e., variables that are either 0 or 1, unless they are absolutely trivial and describe a unit cube themselves. Whenever the problem contains a boolean variable, it is beneficial to skip the unit cube test. This is also the reason why we provide no experimental results for the theory of linear mixed arithmetic, i.e., the few mixed benchmarks available in the SMT-LIB all contain boolean variables.

Summary

We have presented the linear cube transformation (Corollary 4.2.2), which allows us to efficiently determine whether a polyhedron contains a cube of a given edge length. Based on this transformation we have created two tests for linear integer arithmetic: the largest cube test and the unit cube test. Our tests can be integrated into SMT theory solvers without sacrificing the advantages that SMT solvers gain from the incremental structure of subsequent subproblems. Furthermore, our experiments have shown that these tests increase efficiency on certain polyhedra such that previously hard sets of constraints become trivial.

Chapter 5

Computing a Complete Basis for Equalities (Implied by a System of LRA Constraints)

Equalities are a special instance of linear arithmetic constraints. They are useful in simplifying systems of arithmetic constraints [START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF], and they are essential for the Nelson-Oppen style combinations of theories [START_REF] Bruttomesso | Delayed theory combination vs. Nelson-Oppen for satisfiability modulo theories: a comparative analysis[END_REF][START_REF] Nelson | Simplification by cooperating decision procedures[END_REF][START_REF] Oppen | Complexity, convexity and combinations of theories[END_REF]. However, they are also an obstacle for our fast cube tests (see Chapter 4). If an inequality system implies an equality, then it has only a surface and no interior; so our cube tests cannot explore an interior and will certainly fail. In order to expand the applicability of our cube tests, we have to develop methods that find, isolate, and eliminate implied equalities from systems of linear arithmetic constraints.

We can detect the existence of an implied equality by searching for a hypercube in our polyhedron. If the maximal edge length of such a hypercube is zero, then there exists an implied equality. This test can be further simplified. By turning all inequalities into strict ones, the interior of the original polyhedron remains while the surface disappears. If the strict system is unsatisfiable, then the original system has no interior and implies an equality. Moreover, the method generates an implied equality as a proof based on an explanation of unsatisfiability for the strict system.

We are also able to extend the above method into an algorithm that computes an equality basis, i.e., a finite representation of all equalities implied by a satisfiable system of inequalities. For this purpose, the algorithm repeatedly applies the above method to find, collect, and eliminate equalities from our system of constraints. When the system contains no more equalities, then the collected equalities represent an equality basis, i.e., any implied equality can be obtained by a linear combination of the equalities in the basis. The equality basis has many applications. If transformed into a substitution, it eliminates all equalities implied by our system of constraints, which results in a system of constraints with an interior and, therefore, improves the applicability of our cube tests. The equality basis also allows us to test whether a system of linear arithmetic constraints implies a given equality. We even extend this test into an efficient method that computes all pairs of equivalent variables inside a system of constraints. These pairs are necessary for the Nelson-Oppen style combination of theories. This chapter is organized as follows: In Section 5.2, we show how to investigate equalities with the linear cube transformation. We do so by first introducing an efficient method for testing whether a system of linear arithmetic constraints implies a given equality (Section 5.2.1); then, we extend the method so that it computes an equality basis for our system of constraints (Section 5.2.2). In Section 5.3, we describe an implementation of our methods as an extension of Dutertre and de Moura's version of the simplex algorithm, which is integrated in many SMT solvers (see Chapter 2.7 and [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]). The implementation generates justifications and preserves incrementality. The efficient computation of an equality basis can then be used in identifying equivalent variables for the Nelson-Oppen combination of theories (Section 5.4). This is also the first application of the equality basis that we discuss in depth. The second application we present uses the equality basis for the computation of a bounded basis (Definition 2.8.5) and for the detection of bounded and unbounded directions (Section 5.5). The final application we present uses the equality basis for quantifier elimination (Section 5.6). Section 5.7 concludes the chapter with a summary of the presented results.

Related Work and Preliminaries

This chapter is based on two publications with Christoph Weidenbach as co-author [START_REF] Bromberger | Computing a complete basis for equalities implied by a system of LRA constraints[END_REF][START_REF] Bromberger | New techniques for linear arithmetic: cubes and equalities[END_REF]. Only Section 5.5 has never appeared in any publication before this thesis. This chapter focuses like the previous chapter on the geometric interpretation of systems of inequalities. This is the reason why we also use in this chapter polyhedron as an alternative name for systems of inequalities Ax ≤ b. The other geometric objects we are considering are cubes. Their definition can be found in Chapter 2.9. This chapter also focuses on the theory of linear rational arithmetic and not on the more general theory of linear mixed arithmetic. Therefore, we abbreviate in this chapter rational satisfiability/equivalence/entailment with satisfiability/equivalence/entailment. Nonetheless, some of the applications we present verge into the theory of linear mixed arithmetic. To avoid confusion, we always specify the type of solution/satisfiability if it is not rational.

The constraints in this chapter are non-strict inequalities and they are either formatted according to the vector representation, i.e., a T i x ≤ b i (see also Chapter 2.2.1), or the standard representation, a i1 x 1 + . . . + a in x n ≤ b i (see also Chapter 2.2.1). This chapter also deals with equalities a T i x = b i besides non-strict inequalities. However, an equality is just our way of highlighting the set of inequalities {a T i x ≤ b i , -a T i x ≤ -b i } as explained in Chapter 2.2.1. Other types of constraints have to be reduced to non-strict inequalities with the techniques presented in Chapter 2.3. This chapter builds on the basics of linear algebra (Chapter 2.1) and linear arithmetic (Chapter 2.2), on the concept of implied constraints (Chapter 2.5), and on the definitions of (un)bounded and (un)guarded problems and variables (Chapter 2.8). Our example implementation (Section 5.3) also builds on the notions and definitions of standard arithmetic decision procedures for SMT solvers as presented in Chapter 2.7.

There also already exist several methods that find, isolate, and eliminate implied equalities [START_REF] Bjørner | Integrating Decision Procedures for Temporal Verification[END_REF][START_REF] Refalo | Approaches to the incremental detection of implicit equalities with the revised simplex method[END_REF][START_REF] Telgen | Identifying redundant constraints and implicit equalities in systems of linear constraints[END_REF][START_REF] Van | Standard forms for rational linear arithmetic in constraint logic programming[END_REF]. Hentenryck and Graf [START_REF] Van | Standard forms for rational linear arithmetic in constraint logic programming[END_REF] define unique normal forms for systems of linear constraints with non-negative variables. To compute a normal form, they first eliminate all implied equalities from the system. To this end, they determine the lower bound for each inequality by solving one linear optimization problem. Similarly, Refalo [START_REF] Refalo | Approaches to the incremental detection of implicit equalities with the revised simplex method[END_REF] describes several incremental methods that use optimization to turn a satisfiable system of linear constraints in "revised solved form" into a system without any implied equalities. Rueß and Shankar also use this optimization scheme to determine a basis of implied equalities [START_REF] Rueß | Solving linear arithmetic constraints[END_REF]. Additionally, they present a necessary but not sufficient condition for an inequality to be part of an equality explanation. During preprocessing, all inequalities not fulfilling this condition are eliminated, thus, reducing the number of optimization problems their method has to compute. However, this preprocessing step might be in itself expensive because it relies on a non-trivial fixed-point scheme. The method presented by Telgen [START_REF] Telgen | Identifying redundant constraints and implicit equalities in systems of linear constraints[END_REF] does not require optimization. He presents criteria to detect implied equalities based on the tableau used in the simplex algorithm, but he was not able to formulate an algorithm that efficiently computes these criteria. In the worst case, he has to pivot the simplex tableau until he has computed all possible tableaux for the given system of constraints. Another method that detects implied equalities was presented by Bjørner [START_REF] Bjørner | Integrating Decision Procedures for Temporal Verification[END_REF]. He uses Fourier-Motzkin variable elimination to compute linear combinations that result in implied equalities.

Our methods that detect implied equalities do not require optimization, which is advantageous because SMT solvers are usually not fine-tuned for optimization. Moreover, we defined our methods for a rather general formulation of linear constraints, which allows us to convert our results into other representations, e.g., the tableau-and-bound representation used in Dutertre and de Moura's version of the simplex algorithm (see Section 5.3), while preserving efficiency. Finally, our method efficiently searches for implied equalities. We neither have to check each inequality independently nor do we have to blindly pivot the simplex tableau. This also makes potentially expensive preprocessing techniques obsolete.

From Cubes to Equalities

If a polyhedron implies an equality, then it has only surface points and neither an interior nor a center. There is no way such a polyhedron contains a unit cube and a largest cube has edge length zero and is just a point in the original polyhedron. Equalities are, therefore, a challenge for the applicability of our cube tests (see Chapter 4).

There even exist systems of inequalities that imply infinitely many equalities. For instance, the system consisting of the inequalities -2x 1 +x 2 ≤ -2, x 1 + 3x 2 ≤ 8, and x 1 -2x 2 ≤ -2 has only one rational solution: the point (x 1 , x 2 ) = (2, 2). Therefore, it implies the equalities -2x 1 + x 2 = -2 and x 1 + 3x 2 = 8, and all linear combinations of those two equalities, i.e.,

λ 1 •(-2x 1 +x 2 )+λ 2 •(x 1 +3x 2 ) = λ 1 •(-2)+λ 2 •8 for all λ 1 , λ 2 ∈ Q.
The above example also points us to another fact about equalities: there exists a finite representation of all equalities implied by a system of inequalities-even if the system implies infinitely many equalities.

One such finite representation is the equality basis for a satisfiable system of inequalities Ax ≤ b. An equality basis is a system of equalities D x = c such that all (explicit and implicit equalities) implied by Ax ≤ b are linear combinations of equalities from D x = c . We prefer to represent each equality basis D x = c as an equivalent system of equalities y -Dz = c such that y = (y 1 , . . . , y ny ) T and z = (z 1 , . . . , z nz ) T are a partition of the variables in x, D ∈ Q ny×nz , and c ∈ Q ny . The existence of such an equivalent system of equalities is guaranteed by Gaussian elimination. Moreover, each variable y i appears exactly once in the system y -Dz = c, that is to say, y i appears only in the row y id T i z i = c i . We choose to represent our equality bases in this manner because this form also correlates to a distinct substitution σ D,c y,z that replaces variable y i with c i + d T i z (see also Chapter 2.2.4):

σ D,c y,z := {y i → c i + d T i z : i ∈ {1, . . . , n y }}. The substitution σ D,c
y,z is important because it allows us to eliminate all equalities from Ax ≤ b. We simply apply the substitution σ D,c y,z to Ax ≤ b and receive a new system A z ≤ b that neither contains the variables y nor implies any equalities. 1 And the substitution σ D,c y,z for the equality basis y -Dz = c has even further applications. For instance, we can directly check whether an equality h T x = g is a linear combination of y -Dz = c and, therefore, implied by both Ax ≤ b and y -Dz = c. We simply apply σ D,c y,z to h T x = g and see if it simplifies to 0 = 0. Other applications of σ D,c y,z , e.g., for the Nelson-Oppen style combination of theories, will be discussed in the Sections 5.4-5.6.

Finding Equalities

The first step in computing an equality basis for a polyhedron Ax ≤ b is to detect whether the system contains any equalities. At the beginning of this section, we have already stated a criterion that detects this: Lemma 5.2.1 (Cube-Equality). Let Ax ≤ b be a polyhedron. Then exactly one of the following statements is true:

(1) Ax ≤ b implies an equality h T x = g with h = 0 n , or (2) Ax ≤ b contains a cube with edge length e > 0.

Proof. This proof is a case distinction over the sign of x e for the following slightly simplified version of the largest cube test (see Chapter 4.3.1 for the original definition):

maximize x e subject to Ax + a x e ≤ b, where a i = 1 2 a i 1 .

(

If the maximum objective value is positive, Ax ≤ b contains a cube with edge length e > 0. Therefore, we have to prove that Ax ≤ b contains no equality h T x = g with h = 0 n , which we will do by contradiction. Assume Ax ≤ b contains an equality h T x = g with h = 0 n . Then, by transitivity of the subset relation, the polyhedron consisting of the inequalities h T x ≤ h and -h T x ≤ -g must also contain a cube of edge length e. However, applying the transformation from Corollary 4.2.2 to this new polyhedron results in two contradicting inequalities: h T x ≤ gh 1 • e 2 and -h T x ≤ -gh 1 • e 2 . Thus, (1) and ( 2) cannot hold at the same time.

If the maximum objective value is zero, then Ax ≤ b is satisfiable but contains no cube with edge length e > 0. Therefore, we have to prove that Ax ≤ b contains an equality h T x = g with h = 0. Consider the dual linear program [START_REF] Schrijver | Theory of linear and integer programming[END_REF] of (5.1): minimize

y T b subject to y T A = (0 n ) T , y T a = 1 , where a i = 1 2 a i 1 , y ≥ 0 . (5.2)
Due to strong duality, the objectives of the dual and primal linear programs are equal [START_REF] Schrijver | Theory of linear and integer programming[END_REF]. Therefore, there exists a y ∈ Q m that has objective y T b = 0 and that satisfies the dual (5.2). Since y T a = 1 and a i ≥ 0 and y i ≥ 0 holds, there exists a k ∈ {1, . . . , m} such that y k > 0. By multiplying y T A = (0 n ) T with an x ∈ Q δ (Ax ≤ b) and isolating a T k x, we get:

a T k x = -m i=1,i =k y i y k a T i x .
Using y i ≥ 0, and our original inequalities a T i x ≤ b i , we get a finite lower bound for a T k x: A cube with positive edge length is enough to prove that there exists no implied equality. The actual edge length e of this cube is not relevant. Therefore, we can assume that the edge length e is arbitrarily small. We can even assume that our edge length is so small that we can ignore the different multiples a i 1 and any infinitesimals introduced by strict inequalities. We just have to turn all of our inequalities into strict inequalities. Lemma 5.2.2 (Strict-Cube). Let Ax ≤ b be a polyhedron, where a i = 0 n , b i = (p i , q i ), q i ≤ 0, and b δ i = (p i , -1) be the strict versions of the bounds b i for all i ∈ {1, . . . , m}. Then the following statements are equivalent:

a T k x = -m i=1,i =k y i y k a T i x ≥ -m i=1,i =k
(1) Ax ≤ b contains a cube with edge length e > 0, and (2) Ax ≤ b δ is satisfiable.

Proof. (1) ⇒ (2): If Ax ≤ b contains a cube of edge length e > 0, then Ax ≤ ba is satisfiable, where a i = e 2 a i 1 . By Lemma 2.3.1, we know that there exists a δ ∈ Q such that Ax ≤ p + qδa . Now, let δ = min{a iq i δ : i = 1, . . . , m} . Since a iq i δ ≥ δ , it holds that Ax ≤ pδ 1 m . Since a i = a i 1 > 0 and q i ≤ 0, it also holds that δ > 0. By Lemma 2.3.1, we deduce that Ax < p and, therefore, Ax ≤ b δ holds.

(2) ⇒ (1): If Ax ≤ b δ is satisfiable, then we know by Lemma 2.3.1 that there must exist a δ > 0 such that Ax ≤ p -δ1 m holds. Let a max = max{ a i 1 : i = 1, . . . , m} , δ = δ 2 , and e = δ amax . Then

p i -δ = p i -δ -e 2 a max ≤ b i -e 2 a i 1 .
Thus, Ax ≤ b contains a cube with edge length e > 0.

In case Ax ≤ b δ is unsatisfiable, Ax ≤ b contains no cube with positive edge length and, therefore by Lemma 5.2.1, an equality. In case Ax ≤ b δ is unsatisfiable, the algorithm returns an explanation, i.e., a minimal set C of unsatisfiable constraints a T i x ≤ b δ i from Ax ≤ b δ (see also Chapter 2.5.1). If Ax ≤ b itself is satisfiable, we can extract equalities from this explanation: Lemma 5.2.3 (Equality Explanation). Let Ax ≤ b be a satisfiable polyhedron, where a i = 0 n , b i = (p i , q i ), q i ≤ 0, and b δ i = (p i , -1) for all i ∈ {1, . . . , m}. Let Ax ≤ b δ be unsatisfiable. Let C be a minimal set of unsatisfiable constraints a T i x ≤ b δ i from Ax ≤ b δ . Then it holds for every

a T i x ≤ b δ i ∈ C that a T i x = b i is an equality implied by Ax ≤ b.
Proof. Because of transitivity of the subset and implies relationships, we can assume that Ax ≤ b and Ax ≤ b δ contain only the inequalities associated with the explanation C. Therefore,

C = {a T 1 x ≤ b δ 1 , . . . , a T m x ≤ b δ m }
. By Lemma 2.5.4 and Ax ≤ b δ being unsatisfiable, we know that there exists a y ∈ Q m with y ≥ 0, y T A = (0 n ) T , and y T b δ < 0. By Lemma 2.5.4 and Ax ≤ b being satisfiable, we know that y T b ≥ 0 is also true. By Lemma 2.5.6, we know that y k > 0 for every k ∈ {1, . . . , m}. Now, we use y T b δ < 0, y T b ≥ 0, and the definitions of < and ≤ for Q δ to prove that y T b = 0 and b = p. Since y T b δ < 0, we get that y T p ≤ 0. Since y T b ≥ 0, we get that y T p ≥ 0. If we combine y T p ≤ 0 and y T p ≥ 0, we get that y T p = 0. From y T p = 0 and y T b ≥ 0, we get y T q ≥ 0. Since y > 0 and q i ≤ 0, we get that y T q = 0 and q i = 0. Since q i = 0, b = p.

Next, we multiply y T A = (0 n ) T with an x ∈ Q δ (Ax ≤ b) to get y T Ax = 0. Since y k > 0 for every k ∈ {1, . . . , m}, we can solve y T Ax = 0 for every a T k x and get:

a T k x = -m i=1,i =k y i y k a T i x . Likewise, we solve y T b = 0 for every b k to get: b k = -m i=1,i =k y i y k b i . Since x ∈ Q δ (Ax ≤ b) satisfies all a T i x ≤ b i , we can deduce b k as the lower bound of a T k x: a T k x = -m i=1,i =k y i y k a T i x ≥ -m i=1,i =k y i y k b i = b k , which proves that Ax ≤ b implies a T k x = b k .
Lemma 5.2.3 justifies simplifications on Ax ≤ b δ . We can eliminate all inequalities in Ax ≤ b δ that cannot appear in the explanation of unsatisfiability, i.e., all inequalities a T i x ≤ b δ i that cannot form an equality

a T i x = b i that is implied by Ax ≤ b. For example, if we have an assignment v ∈ Q n δ such that Av ≤ b is true, then we can eliminate every inequality a T i x ≤ b δ i for which a T i v = b i is false.
According to this argument, we can also eliminate all inequalities a T i x ≤ b δ i that were already strict inequalities in Ax ≤ b.

Computing an Equality Basis

We now present the algorithm EqBasis(A x ≤ b ) (Figure 5.1) that computes an equality basis for a polyhedron A x ≤ b . In a nutshell, EqBasis iteratively detects and removes equalities from our system of inequalities and collects them in a system of equalities until it has a complete equality ). This has the following effects: (i) the new system of inequalities A z ≤ b implies no longer the equality a T i z = b i ; and (ii) it no longer contains the variable z k . Next, we apply σ to our system of equalities (line 16) and concatenate the equality

z k + n j=1,j =k a ij a ik z j = b i
a ik to the end of it. This has the following effects: (i) the new system of equalities y -D z = c implies a T i z = b i ; and (ii) the variable z k appears exactly once in y -D z = c . This means that we can now re-partition our variables so that z := (z 1 , . . . , z k-1 , z k+1 , . . . , z n ) T and y l+1 := z k to get two new systems Az ≤ b and y -Dz = c that are equivalent to our original polyhedron (line 20). Finally, we remove all rows 0 ≤ 0 from Az ≤ b because those rows are trivially satisfied but would obstruct the detection of equalities with Lemma 5.2.2.

To prove the correctness of the algorithm, we first need to prove that moving the equality from our system of inequalities to our system of equalities preserves equivalence, i.e, the systems (Az ≤ b) ∪ (y -Dz = c) and (A z ≤ b ) ∪ (y -D z = c ) are equivalent in line 16. Lemma 5.2.4 (Equality Elimination). Let: (i) Az ≤ b be a system of inequalities; (ii) y -Dz = c be a system of equalities; (iii) h T z = g be an equality implied by Az ≤ b with

h k = 0; (iv) σ := {z k → g h k -n j=1,j =k h j h k z j } be a substitution based on this equality; (v) y := (y 1 , . . . , y l , z k ) T and z := (z 1 , . . . , z k-1 , z k+1 , . . . , z n ) T ; (vi) (A z ≤ b ) := (Az ≤ b)σ ; (vii) (y -D z = c ) := (y -Dz = c)σ ∪ {z k + n j=1,j =k h j h k z j = g h k }; (viii) u ∈ Q ny δ , v ∈ Q nz δ ; (ix) u = (u 1 , . . . , u ny , v k ) T and v = (v 1 , . . . , v k-1 , v k+1 , . . . , v nz ) T . Then (Av ≤ b)∪(u-Dv = c) is true if and only if (A v ≤ b )∪(u -D v = c ) is true. Proof. First, we create a new substitution σ v := {z k → g h k -n j=1,j =k h j
h k v j } that is equivalent to σ except that it directly assigns the variables z i to their values v i . Let us now assume that either (Av ≤ b)

∪ (u -Dv = c) or (A v ≤ b ) ∪ (u -D v = c ) is true. This means that h T v = g is also true, either by definition of (Av ≤ b) or (u -D v = c ). But h T v = g is true also implies that v k = g h k -n j=1,j =k h j h k v j is true. Therefore, σ v simplifies to the assignment z k → v k . So (Av ≤ b) ∪ (u -Dv = c) and (A v ≤ b ) ∪ (u -D v = c
) simplify to the same expressions and if one combined system is true, so is the other.

The algorithm EqBasis(A x ≤ b ) decomposes the original system of inequalities A x ≤ b into a reduced system Az ≤ b that implies no equalities, and an equality basis y -Dz = c. The algorithm is guaranteed to terminate because the variable vector z decreases by one variable in each iteration. Note that EqBasis(A x ≤ b ) constructs y -Dz = c in such a way that the substitution σ D,c y,z is the concatenation of all substitutions σ from every previous iteration. Therefore, we also know that σ D,c y,z applied to A x ≤ b results in the system of inequalities Az ≤ b that implies no equalities. We exploit this fact to prove the correctness of EqBasis(A x ≤ b ), but first we need two more auxiliary lemmas. y,z simplifies to 0 = 0. An equality basis is already defined as a set of equalities y -Dz = c that implies exactly those equalities implied by Ax ≤ b. So we only need to prove that h T x = g is implied by y -Dz = c if (h T x = g)σ D,c y,z simplifies to 0 = 0.

Lemma 5.2.6 (Equality Substitution). Let y -Dz = c be a satisfiable system of equalities. Let h T x = g be an equality. Then y -Dz = c implies h T x = g iff (h T x = g)σ D,c y,z simplifies to 0 = 0.

Proof. First, let us look at the case where h T x = g is an explicit equality

y i -d T i z = c i in y -Dz = c. Then (y i -d T i z = c i )σ D,c
y,z simplifies to 0 = 0 because σ D,c y,z maps y i to d T i z + c i and the variables z j are not affected by σ D,c y,z .

Next, let us look at the case where h T x = g is an implicit equality in y -Dz = c. Since y -Dz = c implies h T z = g, y -Dz = c and (y -Dz = c) ∪ (h T z = g) must be equivalent. By Lemma 5.2.5, it follows that (y-Dz = c)σ D,c y,z and ((y-Dz = c)∪(h T z = g))σ D,c y,z are also equivalent. As we stated at the beginning of this proof, (y id T i z = c i )σ D,c y,z simplifies to 0 = 0. An equality h T z = g that simplifies to 0 = 0 is true for all v ∈ Q nz δ . Moreover, only equalities that simplify to 0 = 0 are true for all v ∈ Q nz δ . This means (y -Dz = c)σ D,c y,z is satisfiable for all assignments and, therefore, (h T z = g)σ D,c y,z must simplify to 0 = 0. Finally, let us look at the case where h T x = g is not an equality implied by y -Dz = c. Suppose to the contrary that (h T z = g)σ D,c y,z simplifies to 0 = 0. From the first part of this prove, we know that (y -Dz = c)σ D,c y,z also simplifies to 0 = 0. Therefore, ((y

-Dz = c) ∪ (h T z = g))σ D,c
y,z simplifies to 0 = 0 and is satisfiable for all assignments. Based on Lemma 5.2.4 and transitivity of equivalence, it follows that (y -Dz = c) ∪ (h T z = g) and (y -Dz = c) are equivalent. Therefore, h T z = g is implied by y -Dz = c, which contradicts our initial assumption. With Lemma 5.2.6, we have now all auxiliary lemmas needed to prove that the algorithm EqBasis is correct: Let us now suppose to the contrary of our initial assumptions that A x ≤ b implies an equality h T x = g that y -Dz = c does not imply. Since h T x = g is not implied by y -Dz = c, the output of (h T x = g )σ D,c y,z is an equality h T z = g, where h = 0 nz . This also implies that (

Az ≤ b) ∪ (h T z = g) is the output of ((A x ≤ b ) ∪ (h T x = g ))σ D,c
y,z . By Lemma 5.2.5, Az ≤ b and (Az ≤ b) ∪ (h T z = g) are equivalent. Therefore, Az ≤ b implies the equality h T z = g, which contradicts the condition in line 7 of EqBasis and, therefore, our initial assumptions.

It is not straight forward how to efficiently integrate our method that finds an equality basis into an SMT solver. Therefore, we now explain how to implement our method as an extension of Dutertre and de Moura's version (see Chapter 2.7.1 and [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]) of the dual simplex algorithm [START_REF] Beale | An alternative method for linear programming[END_REF][START_REF] Lemke | The dual method of solving the linear programming problem[END_REF]. Our goal for the extension is that it keeps all benefits of the simplex algorithm, i.e., it stays incremental and produces justifications. Both are properties necessary for an efficient theory solver.

We already outlined the simplex algorithm in Chapter 2.7.1. We also mention in Chapter 2.7.1 that this algorithm uses a different input format than the one we prefer for our theory definitions. We defined the theory for the equality basis by representing our input constraints through inequalities Ax ≤ b because inequalities represent the set of solutions more intuitively. In the simplex algorithm, the input constraints are represented instead in the tableau representation (Chapter 2.4.2). Therefore, the input consists of two sets of variables z 1 , . . . , z n ∈ N and y 1 , . . . , y m ∈ B, a tableau Az = y, and two sets of bound functions L and U, which define a set of bounds L(x j ) ≤ x j ≤ U(x j ) for the variables x j ∈ N ∪B. (We explain the conversion between the two formats also in Chapter 2.4.2.) Due to the change in format, it might seem difficult to efficiently integrate our method in the simplex algorithm. The truth, however, is that the tableau representation grants us several advantages for the implementation of our equality basis method. For example, we do not have to explicitly eliminate variables via substitution, but we do so automatically via pivoting.

Integration in the Simplex Algorithm

The tableau representation of an equality basis is a tableau Az = y and a set of tightly bounded variables, i.e., a set of variables x j such that β(x j ) := L(x j ) or β(x j ) := U(x j ) for all satisfiable assignments β. Therefore, one way of determining an equality basis is to find all tightly bounded variables.

To find all tightly bounded variables, we present a new extension of the simplex algorithm called FindTBnds() (Figure 5.2). This extension uses our Lemmas 5.2.2 & 5.2.3 to iteratively find all bounds L(x j ) ≤ x j (x j ≤ U(x j )) that hold tightly for all satisfiable assignments β, and then turns them into explicit equalities by setting U(x j ) := L(x j ) (L(x j ) := U(x j )). But first of all, FindTBnds() determines if our constraint system is actually satisfiable with a call of Check(). If the system is unsatisfiable, then it has no solutions and implies all equalities. In this case, FindTBnds() returns false.

Otherwise, we get a satisfiable assignment β from Check() and we use this assignment in Initialize() (Figure 5.3) to eliminate all bounds that do not hold tightly under β (i.e., β(x i ) > L(x i ) or β(x i ) < U(x i )). We know that we can eliminate these bounds without losing any tightly bounded va-if there is such a z j then pivot(y i , z j );

12 end 13 end 14 for x k ∈ B ∪ N do 15 if L(x k ) < U(x k ) then 16 if L(x k ) = -∞ then L(x k ) := L(x k ) + δ; 17 if U(x k ) = +∞ then U(x k ) := U(x k ) -δ; 18 if x k ∈ N and L(x k ) > β(x k ) then update(x k , L(x k )); 19 if x k ∈ N and U(x k ) < β(x k ) then update(x k , U(x k ));

3: Initialization function for computing an equality basis in tableau representation

If Check returns true, then the original system of inequalities implies no further tightly bounded variables (Lemma 5.2.2). We exit the loop and revert the bounds of the remaining variables x k with L(x k ) = U(x k ) to their original values. As a result, we have also reverted to a linear system equivalent to our original constraint system. The only difference is that now all tightly bounded variables x k are explicit equalities because L(x k ) = U(x k ). Moreover, the tableau Az = y and the non-basic variables that are tightly bounded represent an equality basis for our original constraint system. The simplex algorithm even represents the current tableau and the tightly bounded non-basic variables in such a way that they also describe Algorithm 14: FixEqs(y i ) Input : A basic variable y i that explains the conflict Effect : Turns the bounds of all variables responsible for the conflict into equalities a substitution σ for the elimination of equalities: the rows of the tableau map each basic variable y i to their row definition z j ∈N a ij z j and the tightly bounded non-basic variables z j , i.e., all variables z j ∈ N with L(z j ) = U(z j ), are mapped to their tight bound L(z j ).

1 for z j ∈ N do 2 if L(z j ) = U(z j ) then 3 if β(y i ) < L(y i ) then 4 if a ij > 0 then 5 U(z j ) := U (z j ); L(z j ) := U (z j ); update(z j , U(z j )); 6 end 7 if a ij < 0 then 8 L(z j ) := L (z j ); U(z j ) := L (z j ); update(z j , L(z j )); 9 end 10 end 11 if β(y i ) > U(y i ) then 12 if a ij > 0 then 13 L(z j ) := L (z j ); U(z j ) := L (z j ); update(z j , L(z j )); 14 end 15 if a ij < 0 then 16 U(z j ) := U (z j ); L(z j ) := U (z j ); update(z j , U(z j ));

Incrementality and Explanations

Note that asserting additional bounds to our system can increase the number of tightly bounded variables. In this case, we have to apply FindTBnds() again to find all tightly bounded variables and to complete the new equality basis. We already mentioned that Check() never pivots a non-basic variable z j into a basic one if L(z j ) = U(z j ) because of the conditions in the lines 6 & 12 of Check(). So the tightly bounded non-basic variables we have computed in the last call to FindTBnds() stay non-basic even if the SMT solver asserts additional bounds for the variables and applies Check() again. Hence, our next application of FindTBnds() does not perform any computations for the tightly bounded variables that were detected by earlier applications of FindTBnds(). This means that our algorithm to compute the equality basis is highly incremental. Another important feature of an efficient SMT theory solver is that it produces good-maybe even minimal-conflict explanations. In a typical SMT solver, a SAT solver based on CDCL (conflict-driven clause-learning) selects and asserts a set of theory literals that satisfy the boolean model. Then the theory solvers verify that the asserted literals that belong to their theory are consistently satisfiable. If the theory solver finds a conflict between the asserted literals, then it returns a conflict explanation. The SAT solver uses the conflict explanation to start a conflict analysis that determines a good point for back jumping so it can select a new set of theory literals. Naturally, a good conflict explanation greatly enhances the conflict analysis and, therefore, the remaining search.

The literals asserted in our simplex based theory solver are bounds for our variables. 2 Our algorithm FindTBnds() asserts bounds independently of the SAT solver. This leads to problems in the conflict analysis because the conflict explanation is no longer comprehensible for the SAT solver. Hence, we have to extend FindTBnds() so it produces justifications (for the bounds it asserts in FixEqs(y i )) in the form of clauses that the SAT solver can comprehend and learn.

We only need to justify bounds asserted by FixEqs(y i ) because all other bounds asserted by FindTBnds() are reverted in line 7 to their original bounds x k ≥ L (x k ) and x k ≤ U (x k ). And even in FixEqs(y i ), we only have to justify the bounds x k ≤ L (x k ) (x k ≥ U (x k )) that make the tight bounds x k ≥ L (x k ) (x k ≤ U (x k )) explicit. We also see that the bounds asserted by FixEqs(y i ) are just linear combinations of existing bounds if we look again at the proof of Lemma 5.2.3. The proof also shows that we can derive this linear combination from the conflict explanation C of the strict system. For instance, if the call to Check() from line 3 of FindTBnds() exits in line 7 with (false, y i ), then the conflict explanation is

C = {y i > L (y i )} ∪{z j < U (z j ) : z j ∈ N and a ij > 0} ∪{z j > L (z j ) : z j ∈ N and a ij < 0}. [57]
If the call to Check() exits instead in line 13 with (false, x i ), then the conflict explanation is

C = {y i < U (y i )} ∪{z j > L (z j ) : z j ∈ N and a ij > 0} ∪{z j < U (z j ) : z j ∈ N and a ij < 0}. [57]

Finding Valid Equations Between Variables

We can efficiently find all valid equations between variables as needed for the Nelson-Oppen style combination of theories if we first apply FindTBnds().

Then, we use the substitution σ that we get from the tableau and the tightly bounded variables to get a normalized term that represents each variable x i .

If the variable z j is non-basic and tightly bounded (i.e., L(z j ) = U(z j )), then the normalized term is the constant value L(z j ). If the variable z j is nonbasic and not tightly bounded (i.e., L(z j ) = U(z j )), then the normalized term is the variable z j itself. If the variable y i is basic, then the normalized term is

z j ∈N ,L(z j ) =U (z j ) a ij z j + z j ∈N ,L(z j )=U (z j ) a ij L(z j )
, where all basic mathematical operations between constant values are replaced by the results of those operations.

We know from Lemma 5.2.6 that x i σ = x k σ simplifies to 0 = 0 if σ is the substitution we get from an equality basis and x i = x k is implied by our constraints. Therefore, both x i σ and x k σ must be represented by the same normalized term if x i and x k are equivalent. So the equality basis together with a normalization procedure has turned semantic equivalence into syntactic equivalence. It is very easy to find variables x i represented by the same normalized term if we store these terms in a DAG, which most SMT solvers already provide for assigning slack variables during the transformation into tableau representation (see Chapter 2.4.2).

Nelson-Oppen Justifications

If we use the equality basis computed by the function FindTBnds() for a Nelson-Oppen style combination of theories, then we also assert equalities x i = x k for all pairs of equivalent variables x i , x k . Since these equalities have been asserted independently of the SAT solver, we have to justify these assertions to the SAT solver.

We get these justifications by looking at the normalized representations of the variables x i and x k that are equivalent. The current set of non-basic variables defines a basis and, therefore, already on its own a normalized representation for all variables. Since this normalized representation only depends on the current tableau Az = y, it is also independent of any of the asserted bounds. The normalized representation we use for the Nelson-Oppen style combination is only an extension of this representation by the tight bounds z j = c j of all tightly bounded non-basic variables. Therefore, the equality x i = x k is implied by those tight bounds z j = c j that were actively used to compute this representation. For instance, if z i and z k are both non-basic and equivalent, then both variables must be tightly bounded so that z i = z k = v. Otherwise, they cannot have the same normalized representation. Therefore, z i = v and z k = v imply z i = z k , or as a clause:

(z i < v ∨ z i > v) ∨ (z k < v ∨ z k > v) ∨ (z i = z k ) .
Next, we look at the case where two basic variables y i and y k are equivalent. But before we give the complete formal justification, let us look at an example. Let the variables z 1 , z 2 , z 3 , z 4 , z 5 be non-basic and the variables y 1 and y 2 be basic. In this example, the basic variables are defined by the non-basic variables as follows:

y 1 = 2z 1 -z 2 + 3z 4 and y 2 = 2z 1 -z 2 + z 5 .
Moreover, let the variables z 2 , z 3 , z 4 , and z 5 be tightly bounded such that z 2 = 1, z 3 = 0, z 4 = 1, and z 5 = 3. If we now replace the tightly bounded non-basic variables, in the definitions of y 1 and y 2 we get that both of their normalized representations are 2z 1 and we have actively used the tight bounds z 2 = 1, z 4 = 1, z 5 = 3 to compute this normalization. Hence, y 1 = y 2 is implied by the tight bounds z 2 = 1, z 4 = 1, and z 5 = 3. The variables y 1 and y 2 are also equivalent if we had not asserted that z 2 = 1 because the normalized representation of both variables without z 2 = 1 is 2z 1z 2 . Hence, z 2 = 1 is redundant for the justification and y 1 = y 2 is also implied by just the tight bounds z 4 = 1 and z 5 = 3.

To find which tightly bounded variables are redundant, we can just look at the coefficients. If a ij and a kj are the same, then any tight bound z j = c j is redundant in the justification. This gives us the following clause as a general justification:

z j ∈N ,L(z j )=U (z j ),a ij =a kj z j < L(z j ) ∨ z j > U(z j ) ∨ y i = y k (5.3)
From this clause, we also get the justification for the case where y i is basic and z k non-basic. We simply treat z k as if it were defined as a basic variable by itself (z k = 1 • z k ), so a kk = 1 and all other a kj = 0. If we simplify these restrictions in the clause justification (5.3) for the case with two basic variables, then we receive the following general justification for the mixed case:

z j ∈N \{z k },L(z j )=U (z j ),a ij =0 z j < L(z j ) ∨ z j > U(z j ) ∨ z j ∈{z k },L(z j )=U (z j ),a ij =1 z j < L(z j ) ∨ z j > U(z j ) ∨ y i = z k
All literals in the above clauses except for the variable equivalences x i = x k are asserted as unsatisfiable in the current model of our SAT solver. This holds because these literals contain only tightly bounded variables. Hence, the SAT solver can assert the literal x i = x k on its own through unit propagation. Note also that all justifications we defined are in some sense minimal: each of the above clauses is a tautology and, if we remove one literal from the clause, then it is no longer a tautology. This fact is another property that enhances any potential conflict analysis.

Application: Exploration of (Un)Bounded Directions

Another application of the equality basis lies in the exploration of bounded and unbounded directions 3 . In this section, we present techniques based on the equality basis that (i) find/detect (un)bounded directions and (ii) compute a bounded basis (Definition 2.8.5), i.e., a finite representation of all bounded directions in a system of inequalities. In Chapter 6, we use these methods to extend branch-and-bound into a complete decision procedure for linear mixed arithmetic.

For the exploration of (un)bounded directions, we exploit connections between the two inequality systems Ax ≤ b and Ax ≤ 0 m . Both systems imply only inequalities that are related to one another because both systems have the same coefficients (see Lemma 2.5.5). For instance, if Ax ≤ b implies h T x ≤ g, then Ax ≤ 0 m also implies an inequality h T x ≤ g , and vice versa. This means that both polyhedra also share the same bounded directions h. However, Ax ≤ 0 m has an advantage over Ax ≤ b. If we look again at Lemma 2.5.5, then we see that Ax ≤ 0 m also implies h T x ≤ 0 whenever it implies an inequality h T x ≤ g. This leads us to the conclusion that all bounded directions h in Ax ≤ 0 m -and, therefore, also in Ax ≤ bcorrespond to implied equalities h T x = 0 in Ax ≤ 0 m . Lemma 5.5.1 (Bounds and Equalities).

Let Q δ (Ax ≤ b) = ∅. Then h is bounded in Ax ≤ b iff Ax ≤ 0 m implies that h T x = 0.
Proof. By Definition 2.8.1, h is bounded in Ax ≤ b means that there exists l, u ∈ Q δ such that Ax ≤ b implies h T x ≤ u and -h T x ≤ -l. By Lemma 2.5.5, this is equivalent to: there exist l, u ∈ Q δ , y, z ∈ Q m with y, z ≥ 0 m , and y T A = h T = -z T A so that y T b ≤ u and z T b ≤ -l. Symmetrically, Ax ≤ 0 implies that h T x = 0 is equivalent to: there exist a y, z ∈ Q m with y, z ≥ 0 m and y T A = h T = -z T A so that y T 0 m ≤ 0 and z T 0 m ≤ 0. Since u and l only have to exist, we can trivially choose them as u := y T b and l := -z T b. This means that y T b ≤ u, z T b ≤ -l, y T 0 m ≤ 0, and z T 0 m ≤ 0 are all trivially satisfied by any pair of linear combinations y, z ∈ Q m with y, z ≥ 0 m such that y T A = h T = -z T A. Hence, the two definitions are equivalent and our lemma holds.

In Section 5.2, we presented our method that finds an equality basis Dx = c for any inequality system. Together with the above lemma, we can also use it to compute a bounded basis for any inequality system Ax ≤ b: we simply compute an equality basis Dx = 0 for Ax ≤ 0 m and D is our bounded basis.

We can then use the bounded basis to determine whether a direction h is bounded in Ax ≤ b. A direction h is bounded in Ax ≤ b if and only if it is a linear combination of the row vectors in D, i.e., there exist a y ∈ Q n such that y T D = h T .

Application: Quantifier Elimination

The methods and theories presented in this chapter have a surprisingly wide range of application. Besides our original goal of investigating equalities, we have also already presented applications for theory combinations with the Nelson-Oppen method and for the exploration of (un)bounded directions. In this section, we show that our methods around equalities are also useful for quantifier elimination.

In general, a quantifier elimination (QE) procedure takes a formula ∃y.φ(y), where φ(y) itself is quantifier-free but may contain extra variables x called parameters, and returns an equivalent formula φ that is quantifierfree. Quantifier elimination procedures typically eliminate the quantifier ∃y by introducing a case distinctions over the values the variable y can assume.

Linear virtual substitution is a complete QE procedure for the theory of linear rational arithmetic [START_REF] Loos | Applying linear quantifier elimination[END_REF]. It eliminates the variable y by creating a case distinction4 exploiting the following fact: a linear rational arithmetic formula φ(y) is rationally satisfiable if and only if φ(l) is satisfiable, where l is the strictest lower bound (or upper bound) of y, i.e., the smallest value for y in any solution to the problem. This value is either represented by -∞ (+∞) or one of the inequalities in φ(y) containing y. There are only finitely many inequalities in φ(y), so by a case distinction over all inequalities containing y satisfiability can be preserved:

∃y ∈ Q.φ(y) ≡ φ(-∞) ∨ a iy y+a T i x≤b i in φ(y) with a iy <0 φ -b i a iy + a T i x a iy .
This case distinction is the source of the worst case doubly exponential complexity of the procedure in case of quantifier alternations. At the same time, there are also instances that we can resolve without case distinctions. For instance, if the formula φ(y) implies an equality h y • y + h T x = g where h y = 0, then we already know one guaranteed definition for the strictest lower bound of y: g hy -h T x hy . A quantifier-free formula that is equivalent to the original one is simply:

∃y ∈ Q.φ(y) ≡ φ g hy -h T x
hy . This technique is well-known and integrated in many QE implementati-ons [START_REF] Dolzmann | Real quantifier elimination in practice[END_REF][START_REF] Loos | Applying linear quantifier elimination[END_REF][START_REF] Sturm | Real quadratic quantifier elimination in risa/asir[END_REF]. Even so, we are unaware of any implementation that makes use of non-explicit equalities for this purpose. This is where our methods that find implicit equalities come into play. Our methods are applicable because QE procedures typically keep φ in a disjunctive form and the respective disjuncts contain often only conjuncts of inequalities. This allows us to efficiently search for an equality.

A similar trick also exists for the theory of linear integer/mixed arithmetic, i.e., for eliminating an integer variable. We only have to additionally verify that the lower bound of the integer variable is an actual integer value. which we can do through one additional divisibility constraint. Therefore,

φ g hy -h T x hy ∪ {(h y |g + h T x)}
is a quantifier-free formula that is equivalent to ∃y ∈ Z.φ(y) if φ(y) implies the equality h y • y + h T x = g where h y = 0. Note that our methods detect only equalities implied by rational solutions of Ax ≤ b, i.e., only those equalities h T x = g that are satisfied by all solutions x ∈ Q δ (Ax ≤ b). There might be integer/mixed entailed equalities that are not generally implied, which means that they cannot be computed with our equality basis method.

Summary

We have presented in Chapter 4 the linear cube transformation (Corollary 4.2.2), which allows us to efficiently determine whether a polyhedron contains a cube of a given edge length. One obstacle for our cube tests are equalities. Resolving these obstacles led to an additional application of the linear cube transformation: investigating equalities. Through Lemmas 5.2.2 & 5.2.3, we have presented a method that efficiently checks whether a system of linear arithmetic constraints implies an equality at all. We use this method in the algorithm EqBasis(Ax ≤ b) to compute an equality basis y -Dz = c, which is a finite representation of all equalities implied by the inequalities Ax ≤ b.

Chapter 6 A Reduction (from Unbounded Linear Mixed Arithmetic Problems) into Bounded Problems

We have already presented two complete decision procedures for linear integer arithmetic: branch-and-bound extended by a priori bounds (Chapter 2.8) and CutSat++ (Chapter 3).

A priori bounds complete branch-and-bound (see Chapter 2.7.3) by extending the input problem with approximated variable bounds -g ≤ x i ≤ g for every variable x i . Adding the variable bounds creates an equisatisfiable problem because the bounds -g ≤ x i ≤ g are approximated large enough, i.e., the original problem has an integer/mixed solution within the bounds whenever it has one outside the bounds. Since the a priori bounds bound all variables, they also reduce the search space for a branch-and-bound solver (and many other integer/mixed arithmetic decision procedures) to a finite search space. So branch-and-bound is guaranteed to terminate. However, these bounds are so large that the resulting search space cannot be explored in reasonable time (e.g. the life time of earth) even for small problems. One reason why the a priori bounds are so impractically large is that they only take parameter sizes into account and not actually the structure of each problem. Therefore, the approximated bound values are by far larger than is necessary.

CutSat++ is a CDCL style algorithm combined with a lazy quantifier elimination procedure. This combination allows us to use a much more structure-oriented approach for completeness. The price for this result is an algorithm that is by far more complicated than the branch-and-bound approach. In particular, (i) it has to consider divisibility constraints in addition to inequalities; (ii) CutSat++ is missing several features that make CDCL style SAT solving so efficient in practice, e.g., exhaustive propagation cannot be done in CutSat++ or we encounter propagation divergence; (iii) it is very inflexible, i.e., there is not much room to improve its search strategy because of its strict termination strategy; and (iv) an extension to mixed arithmetic would make it even more complex and make it even more inflexible.

Due to their disadvantages, neither a priori bounds or CutSat++ have been integrated in any state-of-the-art SMT solvers [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF]. As far as we know, none of the state-of-the-art SMT solvers use any method that guarantees termination for linear integer or mixed arithmetic. Therefore, they are all incomplete.

In this chapter, we present a different approach to reach a complete decision procedure. Like a priori bounds, our approach consists of mixed equisatisfiable transformations that turn every problem into a bounded problem. This means that we can also use them to make branch-and-bound (and many other integer/mixed arithmetic decision procedures) complete. In contrast to a priori bounds, our transformations do not use approximations, which also means that we never have to deal with over-approximation and the resulting loss of efficiency. Instead, our transformations orient themselves on the structure of the input problem by eliminating the unbounded directions in the problem. This sounds similar to CutSat++, but our transformations manage to avoid the shortcomings of the CutSat++ calculus. This means that (i) they do not introduce new types of constraints; (ii) the combination of branch-and-bound and our transformations stays flexible; and (iii) the extension to mixed arithmetic is intuitive and requires no major changes. As a result, our approach is efficient in practice and we can even support this claim with benchmark experiments.

We already explained in Chapter 2.8 that the (un)boundedness of a problem indicates whether branch-and-bound terminates. For this analysis, we partitioned our input problems into four categories: guarded problems, unguarded but bounded problems, absolutely unbounded problems, and partially unbounded problems. Guarded problems and unguarded but bounded problems compose together all bounded problems, i.e., all problems for which branch-and-bound has a finite search space and, therefore, always terminates. Absolutely unbounded problems are unbounded, which causes some problems for branch-and-bound alone. The truth is, however, that they are quite trivial because they always have an integer solution. In Chapter 4, we described two cube tests that detect and solve absolutely unbounded problems in polynomial time. Therefore, branch-and-bound becomes also complete for absolutely unbounded problems if we extended it with our cube tests. The actual difficult case is when some directions are bounded and others unbounded, i.e., when the problem is partially unbounded. Here, branch-and-bound and most other algorithms diverge or become inefficient in practice. The transformations, which we present, are designed to ef-ficiently handle this category of problems. Our approach is also efficient because we only actually apply our transformations on partially unbounded problems. We avoid the application to the other types of problems because we use the method from Chapter 5.5 to efficiently determine the type of (un)boundedness of our input problems.

The contributions of this chapter are as follows: We present satisfiability preserving transformations that reduce unbounded problems into bounded problems. On these bounded problems, most linear mixed decision procedures become terminating, which we show on the example of branch-andbound. The transformations are, therefore, extensions that complete other decision procedures. Our transformations work by eliminating unbounded variables. First, we use the Double-Bounded reduction (Section 6.3) to eliminate all unbounded inequalities from our constraint system. Then we use the Mixed-Echelon-Hermite transformation (Section 6.2) to shift the variables of our system to ones that are either bounded or do not appear in the new inequalities and are, therefore, eliminated. With Corollary 6.2.9 & Lemma 6.3.5 we explain how to efficiently convert certificates of (un)satisfiability between the transformed and the original system. Our method is efficient because it is fully guided by the structure of the problem. This is confirmed by experiments (Section 6.5). Finally, we explain in Section 6.4 how to implement the presented procedures in an incrementally efficient way. This is relevant for an efficient SMT implementation.

Related Work and Preliminaries

This chapter is based on the publication [START_REF] Bromberger | A reduction from unbounded linear mixed arithmetic problems into bounded problems[END_REF] and focuses on the theory of linear mixed arithmetic. Nonetheless, we always specify the type of solution/satisfiability. Only implications are always assumed to be rational/general entailments (see Chapter 2.5).

The constraints in this chapter are non-strict inequalities and they are either formatted according to the vector representation, i.e., a T i x ≤ b i (see also Chapter 2.2.1), or the standard representation, a i1 x 1 + . . . + a in x n ≤ b i (see also Chapter 2.2.1). Other types of constraints have to be reduced to non-strict inequalities with the techniques presented in Chapter 2.3. This chapter builds on the basics of linear algebra (Chapter 2.1) and linear arithmetic (Chapter 2.2), on the concept of implied constraints (Chapter 2.5), and on the definitions of (un)bounded and (un)guarded problems and variables (Chapter 2.8). Our incremental implementation (Section 6.4) also builds on the notions and definitions of standard arithmetic decision procedures for SMT solvers as presented in Chapter 2.7.

There also exist several publications by other authors that are highly relevant to the contributions presented in this chapter. The first related work is by Bobot et al. [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF]. They present in their paper [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF] a complete decision procedure for linear integer arithmetic, which is also the most similar approach to our transformations that we found in the literature. The decision procedure by Bobot et al. dynamically eliminates one linear independent bounded direction at a time via transformation. For comparison, our own transformations statically eliminate all unbounded directions at once. The disadvantage of the dynamic approach is that it is very restrictive and does not leave enough freedom to change strategies or to add complementing techniques. Moreover, Bobot et al.'s implementation of the decision procedure in the SMT solver Ctrl-Ergo uses this transformation approach for all problems and not only the partially unbounded ones, which sometimes leads to a massive overhead on bounded problems.

The second related work is by Christ and Hoenicke [START_REF] Christ | Cutting the mix[END_REF]. They present in their paper [START_REF] Christ | Cutting the mix[END_REF] an extension to branch-and-bound that uses the Mixed-Echelon-Hermite transformation to find more versatile branches and cuts. Since one of our proposed transformations is also the Mixed-Echelon-Hermite transformation, there exist some interesting similarities and relationships between their approach and our approach. For instance, our Double-Bounded reduction alone would be sufficient to make Christ and Hoenicke's extension complete.

The third related work is by Philipp Rümmer [START_REF] Rümmer | A constraint sequent calculus for first-order logic with linear integer arithmetic[END_REF]. He presents in his paper [START_REF] Rümmer | A constraint sequent calculus for first-order logic with linear integer arithmetic[END_REF] a constraint sequent calculus that is complete for quantified linear integer arithmetic. It terminates because it simulates in the worst case the complete and terminating Omega Test [START_REF] Pugh | The omega test: a fast and practical integer programming algorithm for dependence analysis[END_REF]. Moreover, it can be extended so it produces interpolants for ground formulas [START_REF] Brillout | An interpolating sequent calculus for quantifier-free presburger arithmetic[END_REF]. This means Rümmer's calculus performs tasks that are more general than our intended task: determining satisfiability of systems of inequalities. This broader focus also leads to one big disadvantage: Rümmer's calculus performs our intended task less efficiently than more specialized methods. This is especially true for determining satisfiability of unbounded problems (see Section 6.5).

Mixed-Echelon-Hermite Transformation

Our goal is to present an equisatisfiable transformation that turns any constraint system into a system that is bounded, i.e., a system on which branchand-bound and many other arithmetic decision procedures terminate. In this section, we only present such a transformation for a subset of constraint systems, which we call double-bounded constraint systems. We then show in the next section that each constraint system can be reduced to an equisatisfiable double-bounded system. We also show how to efficiently transform a mixed solution from the double-bounded reduction to a mixed solution for the original system. Definition 6.2.1 (Double-Bounded Constraint System). A constraint system Dx ≤ u is double-bounded if Dx ≤ u implies Dx ≥ l for l ∈ Q m . For such a double-bounded system, we call the bounds u the upper bounds of Dx and the bounds l the lower bounds of Dx. Moreover, we typically write l ≤ Dx ≤ u instead of Dx ≤ u although the lower bounds l are only implicit.

Note that only the inequalities in a double-bounded constraint system are guaranteed to be bounded. Variables might still be unbounded. For instance, in the constraint system 1 ≤ 3x 1 -3x 2 ≤ 2 both inequalities are bounded but the variables x 1 and x 2 are not. Moreover, the above constraint system is also an example where branch-and-bound diverges (see Example 2.8.3 in Chapter 2.8). This means that even bounding all inequalities does not yet guarantee termination. So for our purposes, a double-bounded constraint system is still too complex.

This changes, however, if we also require that the coefficient matrix D of our constraint system is a lower triangular matrix with gaps: Definition 6.2.2 (Lower Triangular with Gaps). A matrix A ∈ Q m×n is lower triangular with gaps if it holds for each column j that piv(A, j) > m or that piv(A, j) < piv(A, k) for all columns k with j < k ≤ n, i.e., column j either has only zero entries or all pivoting entries right of j have a higher row index.

A matrix is lower triangular if and only if the row indices of its pivots are strictly increasing, i.e., piv(A, 1) < . . . < piv(A, n). If we also allow it to have gaps, only the row indices of pivots with non-zero columns have to be strictly increasing. Now we get termination for free because of our restrictions: Lemma 6.2.3 (Lower Triangular Double-Bounded Systems). Let the matrix D ∈ Q m×n be lower triangular with gaps and l ≤ Dx ≤ u be a doublebounded constraint system. Then each variable x j is either bounded, i.e., l ≤ Dx ≤ u implies that l j ≤ x j ≤ u j , or its column in D has only zero entries.

Proof. Proof by induction. Assume that the above property already holds for all variables x k with k < j. Let p = piv(D, j). If p > m, then the column j of D is zero and we are done. If p ≤ m, then the pivoting entry d pj of column j is non-zero. Because of Definition 6.2.2 and our induction hypothesis, this also means that each column k with k < j has either a zero entry in row p or the variable x k is bounded by our induction hypothesis, i.e., l ≤ Dx ≤ u implies l k ≤ x k ≤ u k . Since Definition 6.2.2 also implies that row p has only zero entries to the right of d pj , the row p has only one unbounded variable with a non-zero entry, viz., x j . This means we can transform the row l p ≤ d T p x ≤ u p into the following two inequalities: l p -j-1 k=1 d pk x k ≤ d pj x j and u p -j-1 k=1 d pk x k ≥ d pj x j , where the variables x k on the left sides are either bounded or d pk = 0. Hence, we can derive an upper and lower bound for x j via bound propagation/refinement (see Chapter 2.7.2). Corollary 6.2.4 (BnB-LTDB-Termination). Branch-and-bound terminates on every double-bounded system l ≤ Dx ≤ u where D is lower triangular with gaps.

Our next goal is to efficiently transform every double-bounded system l ≤ Dx ≤ u into an equisatisfiable system that also has a lower triangular coefficient matrix with gaps. We start by defining a class of transformations that do not only preserve mixed equisatisfiability, but are also very expressive. Definition 6.2.5 (Mixed Column Transformation Matrix [START_REF] Christ | Cutting the mix[END_REF]). Given a mixed constraint system. A matrix V ∈ Q n×n is a mixed column transformation matrix if it is invertible and consists of an invertible matrix

V (Q) ∈ Q n 1 ×n 1 , a unimodular matrix V (Z) ∈ Z n 2 ×n 2 , and a matrix V (M ) ∈ Q n 1 ×n 2 such that V = V (Q) V (M ) 0 n 2 ×n 1 V (Z)
.

The formal definition of mixed column transformation matrices may seem anything but intuitive. However, they actually describe a straightforward class of transformations, viz., any combination of mixed equisatisfiable column transformations that can be performed on a matrix A. Mixed equisatisfiable column transformations are either (i) multiplying a column by -1; (ii) the swapping of two columns i, j of the same type, i.e., i, j ≤ n 1 or i, j > n 1 ; (iii) multiplying a rational column j (i.e., j ≤ n 1 ) with a non-zero rational factor; (iv) adding a rational multiple of a column j with j ≤ n 1 to any other column i; and (v) adding an integer multiple of a column j > n 1 to a different column i with i > n 1 . If we perform the same mixed equisatisfiable column transformations that resulted in H from A to an n × n identity matrix, then the transformed identity matrix V is a mixed column transformation matrix and H = AV . We just compacted the column transformations into a matrix. We cannot just use V to redo the column transformations, but we can also use its inverse to undo them: Lemma 6.2.6 (Mixed Column Transformation Inversion [START_REF] Christ | Cutting the mix[END_REF]). Given a mixed constraint system. Let V ∈ Q n×n be a mixed column transformation matrix. Then V -1 is also a mixed column transformation matrix. This means that each mixed column transformation matrix defines a bijection from (Q n 1 × Z n 2 ) to (Q n 1 × Z n 2 ). Hence, they guarantee mixed equisatisfiability: Lemma 6.2.7 (Mixed Column Transformation Equisatisfiability [START_REF] Christ | Cutting the mix[END_REF]). Let Ax ≤ b be a mixed constraint system. Let V ∈ Q n×n be a mixed column transformation matrix. Then (AV )y ≤ b and Ax ≤ b are mixed equisatisfiable and every solution y ∈ M((AV )y ≤ b)) can be converted into a solution V y = x ∈ M(Ax ≤ b) and vice versa.

Moreover, the mixed column transformation matrix V also establishes a direct relationship between the linear combinations of the original constraint system and the transformed one:

Lemma 6.2.8 (Mixed Column Transformation Implications). Let Ax ≤ b be a constraint system. Let V ∈ Q n×n be a mixed column transformation matrix. Let Ax ≤ b imply h T x ≤ g. Then AV z ≤ b implies h T V z ≤ g.
Proof. By Lemma 2.5.5, Ax ≤ b implies h T x ≤ g iff there exists a nonnegative linear combination y ∈ Q n such that y ≥ 0, y T A = h T and y T b ≤ g. Multiplying y T A = h T with V results in y T AV = h T V and thus y is also the non-negative linear combination of inequalities AV z ≤ b that results in h T V z ≤ g. Corollary 6.2.9 (Mixed Column Transformation Certificates). Let Ax ≤ b be a constraint system. Let V ∈ Q n×n be a mixed column transformation matrix. Then y is a certificate of unsatisfiability 1 for Ax ≤ b iff it is one for AV z ≤ b. Now we only need a mixed column transformation matrix V for every coefficient matrix A such that H = AV is lower triangular with gaps. One such matrix V is the one that transforms A into Mixed-Echelon-Hermite normal form: Definition 6.2.10 (Mixed-Echelon-Hermite Normal Form [START_REF] Christ | Cutting the mix[END_REF]). A matrix

H ∈ Q m×n is in Mixed-Echelon-Hermite normal form if H = E 0 r×(n 1 -r) 0 r×n 2 E 0 (m-r)×(n 1 -r) H ,
where E is an r × r identity matrix (with r ≤ n 1 ), E ∈ Q (m-r)×r , and H ∈ Q (m-r)×n 2 is a matrix in hermite normal form, i.e., a lower triangular matrix without gaps, where each entry h pk in the row p = piv(H , j) is non-negative and smaller than h pj ).

The following proof for the existence of the Mixed-Echelon-Hermite normal form is constructive and presents the Mixed-Echelon-Hermite transformation. Lemma 6.2.11 (Mixed-Echelon-Hermite Transformation). Let A ∈ Q m×n be a matrix, where the upper left r × n 1 submatrix has the same rank r as the complete left m × n 1 submatrix. Then there exists a mixed transformation matrix V ∈ Q n×n such that H = AV is in Mixed-Echelon-Hermite normal form.

Proof. Proof from [START_REF] Christ | Cutting the mix[END_REF] with slight modifications so it also works for singular matrices. We subdivide A into

A = A 11 A 12 A 21 A 22 such that A 11 ∈ Q r×n 1 , A 12 ∈ Q r×n 2 , A 21 ∈ Q m-r×n 1 , and A 21 ∈ Q m-r×n 2 .
Then we bring A 11 with an invertible matrix

V 11 ∈ Q n 1 ×n 1 into reduced echelon column form H 11 = (E 0 r×(n 1 -r) ) = A 11 V 11
, where E is an r × r identity matrix. We get V 11 and H 11 by using Bareiss algorithm instead of the better known Gaussian elimination as it is polynomial in time [START_REF] Bareiss | Sylvester's identity and multistep integer-preserving Gaussian elimination[END_REF]. 2 Note that the last n 1r columns of H 21 = (H 21 0 (m-r)×(n 1 -r) ) = A 21 V 11 are also zero because all rows in A 21 are linear dependent of A 11 (due to the rank). Next we notice that

A 12 -A 11 V 11 A 12 0 (n 1 -r)×n 2 = A 12 -(E 0 r×(n 1 -r) )
A 12 0 (n 1 -r)×n 2 = 0 r×n 2 so we can reduce the upper right submatrix A 12 to zero by adding multiples of the n 1 columns with rational variables to the n 2 columns with integer variables. However, this also transforms the lower right submatrix A 22 into

H 22 = A 22 -A 21 V 11 A 12 0 (n 1 -r)×n 2 .
Finally, we transform this new submatrix H 22 into hermite normal form H 22 via the algorithm of Kannan and Bachem [START_REF] Kannan | Polynomial algorithms for computing the smith and hermite normal forms of an integer matrix[END_REF] (or a similar polynomial time algorithm). This algorithm also returns a unimodular matrix V 22 ∈ Z n 2 ×n 2 such that H 22 = H 22 V 22 . To summarize: our total mixed transformation matrix is

V =   V 11 -V 11 • A 12 0 (n 1 -r)×n 2 • V 22 0 n 2 ×n 1 V 22   and H = H 11 0 r×n 2 H 21 H 22 .
It is not possible to transform every matrix A ∈ Q m×n into Mixed-Echelon-Hermite normal form. We have to restrict ourselves to matrices, where the upper left r × n 1 submatrix has the same rank r as the complete left m × n 1 submatrix. However, this is very easy to accomplish for a system of linear mixed arithmetic constraints l ≤ Ax ≤ u. The reason is that the order of inequalities does not change the set of satisfiable solutions. Hence, we can swap the inequalities and, thereby, the rows of A until its upper left r × n 1 submatrix has the desired form. This also means that there are usually multiple possible inequality orderings that each have their own Mixed-Echelon-Hermite normal form H.

To conclude this section: whenever we have a double-bounded constraint system l ≤ Dx ≤ u, we can transform it (after some row swapping) into an equisatisfiable system l ≤ Hy ≤ u where H = DV is in Mixed-Echelon-Hermite normal form and V y = x. Since H is also a lower triangular matrix with gaps, branch-and-bound terminates on l ≤ Hy ≤ u with a mixed solution t or it will return unsatisfiable (Corollary 6.2.4). Moreover, we can convert any mixed solution t for l ≤ Hy ≤ u into a mixed solution s for l ≤ Dx ≤ u by setting s := V t. Hence, we have a complete algorithm for double-bounded constraint systems.

Double-Bounded Reduction

In the previous Section, we have shown how to solve a double-bounded constraint system. Now we show how to reduce any system of inequalities A x ≤ b to a mixed equisatisfiable double-bounded system l ≤ Dx ≤ u. Moreover, we explain how to take any solution of l ≤ Dx ≤ u and turn it into a solution for A x ≤ b .

As the first step of our reduction, we reformulate the constraint system into a so-called split system: Definition 6.3.1 (Split System). (Ax ≤ b) ∪ (l ≤ Dx ≤ u) is a split system if: (i) all directions are unbounded in Ax ≤ b; and (ii) all row vectors a i from A are also unbounded in (Ax ≤ b) ∪ (l ≤ Dx ≤ u). Moreover, we call Ax ≤ b the (absolutely) unbounded part and l ≤ Dx ≤ u the (double-)bounded part of the split system.

A split system consists of an (absolutely) unbounded part Ax ≤ b that is guaranteed to have (infinitely many) integer solutions (see Lemma 2.8.8) and a double-bounded part l ≤ Dx ≤ u (see Figures 6.1-6.3 for an example). Any constraint system can be brought into the above form. We just have to move all unbounded inequalities into the unbounded part and all bounded inequalities into the bounded part. Lemma 6.3.2 (Split Equivalence). Let A x ≤ b be a system of inequalities. Then there exists an equivalent split system (Ax ≤ b) ∪ (l ≤ Dx ≤ u) where:

(i)A ∈ Q m×n , A ∈ Q m 1 ×n , and D ∈ Q m 2 ×n so that m 1 + m 2 = m; (ii) all rows d T
i of D and a T k of A appear as rows in A ; and (iii) Dx ≤ u implies l ≤ Dx. Proof. For (i), (ii), and the equivalence, it is enough to move all bounded inequalities a T i x ≤ b i of A x ≤ b into Dx ≤ u and all unbounded inequalities into Ax ≤ b. For (iii), we assume for a contradiction that Dx ≤ u does not imply l i ≤ d T i x but (Dx ≤ u) ∪ (Ax ≤ b) does. By Lemma 2.5.5, this means that there exists a y ∈ Q m 2 with y ≥ 0 m 2 and a z ∈ Q m 1 with z ≥ 0 m 1 so that y T D+z T A = -d T i and y T u+z T b ≤ -l i . We also know that there exists a z k > 0 because Dx ≤ u alone does not imply l i ≤ d T i x. We use this fact to reformulate y

T D+z T A = -d T i into -a T k = 1 z k y T D + d T i + m 1
j=1,j =k z j a T j , and use the bounds of the inequalities in Dx ≤ u and Ax ≤ b to derive a lower bound for a T k x:

-a T k x ≤ 1 z k y T u + u i + m 1 j=1,j =k z j b j .
Hence, a T k is bounded in A x ≤ b and we have our contradiction.

The above Lemma also shows that the bounded part of a constraint system is self-contained, i.e., a constraint system implies that a direction is bounded if and only if its bounded part does. The actual difficulty of reformulating a system into a split system is not the transformation per se, but finding out which inequalities are bounded or not. There are many ways to detect whether an inequality is bounded by a constraint system. Most work even in polynomial time. For instance, solving the linear rational optimization problem "minimize a T i x such that Ax ≤ b" returns -∞ if a i is unbounded, ∞ if Ax ≤ b has no rational solution, and the optimal lower bound l i for a T i x otherwise. However, even with this technique we still need to solve m linear optimization problems to determine for all inequalities a T i x ≤ b i whether they are bounded.

A, in our opinion, more efficient alternative is based on our algorithm for finding equality bases (see Chapter 5). In Lemma 5.5.1 of Chapter 5.5, we have shown that Q δ (Ax ≤ b) = ∅ implies the following equivalence: the direction h is bounded in Ax ≤ b iff Ax ≤ 0 m implies h T x = 0. It is easy and efficient to compute an equality basis for Ax ≤ 0 m and to determine with it the inequalities in Ax ≤ b that are bounded (see also Chapter 5.5). The only disadvantage towards the optimization approach is that we do not derive an optimal lower bound l for the inequalities. This is no problem because only the existence of lower bounds is relevant for our transformations and not the actual bound values.

Although we now know how to transform every system of inequalities into a split system (Ax ≤ b) ∪ (l ≤ Dx ≤ u), we still need to reduce the split system to a mixed equisatisfiable double-bounded system. However, this task is trivial because the unbounded part (Ax ≤ b) is actually inconsequential to the rational/mixed satisfiability of the system. It may reduce the number of rational/mixed solutions, but it never removes them all. Hence, (Ax ≤ b) ∪ (l ≤ Dx ≤ u) is equisatisfiable to just l ≤ Dx ≤ u. We first show this equisatisfiability for the rational case:

Lemma 6.3.3 (Rational Extension). Let (Ax ≤ b) ∪ (l ≤ Dx ≤ u) be a split system. Let s ∈ Q n δ be a rational solution to the bounded part l ≤ Dx ≤ u such that Ds = g, where g ∈ Q m 2 δ . Then (Ax ≤ b) ∪ (Dx = g) has a rational solution s ∈ Q n δ .
Proof. Assume for a contradiction that (Ax ≤ b) ∪ (Dx = g) has no solution. By Lemma 2.5.4, this means that there exist a y ∈ Q m 1 with y ≥ 0 m 1 and z, z ∈ Q m 2 with z, z ≥ 0 m 2 such that y T A + z T Dz T D = (0 n ) T and y T b + z T gz T g < 0. Since Dx = g is satisfiable by itself, there must exist a y i > 0. Now we use this fact to reformulate the equation

y T A + z T D -z T D = (0 n ) T into -a T i = 1 y i m 1 j=1j =i y j a T j + z T D -z T D ,
from which we deduce a lower bound for

a T i x in (Ax ≤ b) ∪ (l ≤ Dx ≤ u): -a T i x ≤ 1 y i m 1 j=1j =i y j b j + z T u -z T l . Therefore, a i is bounded in (Ax ≤ b) ∪ (l ≤ Dx ≤ u), which is a contra- diction.
Note that the bounded part l ≤ Dx ≤ u of a split system can still have unbounded directions (not inequalities). Some of these unbounded directions in l ≤ Dx ≤ u are the orthogonal directions to the row vectors d i , i.e., vectors v j ∈ Z n such that d T i v j = 0 for all i ∈ {1, . . . , m 2 }. This also means that the existence of one mixed solution s ∈ (Q n 1 × Z n 2 ) and one unbounded direction proves the existence of infinitely many mixed solutions. We just need to follow the orthogonal directions, i.e., s = λ • v j + s is also a mixed solution for all λ ∈ Z because

d T i s = λ • d T i v j + d T i s = d T i s.
(See Figures 6.1-6.3 for an example.) In the next two steps, we prove that Ax ≤ b cannot cut off all of these orthogonal solutions because it is completely unbounded. The first step proves that Ax ≤ b remains absolutely unbounded even if we settle on one set of orthogonal solutions, i.e., enforce Dx = Ds for some solution s.

Lemma 6.3.4 (Persistence of Unboundedness). Let (Ax ≤ b)∪(l ≤ Dx ≤ u) be a split system. Let s ∈ Q n δ be a rational solution for l ≤ Dx ≤ u such that Ds = g (with g ∈ Q m 2 δ
). Then all row vectors a i from A are still unbounded in (Ax ≤ b) ∪ (Dx = g).

Proof. By Lemma 6.3.3, (Ax ≤ b) ∪ (Dx = g) has at least a rational solution s * . Moreover, (Ax ≤ 0) ∪ (Dx = 0) does not imply a T i x = 0 because of Lemma 5.5.1 and the assumption that the row vectors a i from A are unbounded in (Ax ≤ b) ∪ (l ≤ Dx ≤ u). In reverse, (Ax ≤ b) ∪ (Dx = g) having a rational solution, (Ax ≤ 0) ∪ (Dx = 0) does not imply a T i x = 0, and Lemma 5.5.1 prove together that the row vectors a i from A are also unbounded in (Ax ≤ b) ∪ (Dx = g).

The next step proves how to extend the mixed solution from the bounded part to the complete system with the help of the Mixed-Echelon-Hermite normal form and the absolute unboundedness of Ax ≤ b.

Lemma 6.3.5 (Mixed Extension). Let (Ax ≤ b) ∪ (l ≤ Dx ≤ u) be a split system. Let s ∈ (Q n 1 δ × Z n 2 ) be a mixed solution for l ≤ Dx ≤ u. Then (Ax ≤ b) ∪ (l ≤ Dx ≤ u) has a mixed solution s ∈ (Q n 1 δ × Z n 2 ). Proof. Let g = Ds.
Without loss of generality we assume that the upper left r × n 1 submatrix of D has the same rank r as the complete left m 1 × n 1 submatrix of D. (Otherwise, we just reorder the rows accordingly.) Therefore, there exists a mixed column transformation matrix V such that H = DV is in Mixed-Echelon-Hermite normal form (see Lemma 6.2.11). By Lemma 6.2.7, there exists a mixed vector t ∈ (Q n 1 δ × Z n 2 ) such that s = V t and t is a mixed-solution to l ≤ Hy ≤ u as well as Hy = g. Let the set B contain all column indices j in H that correspond to bounded variables y j . Let the set U contain all column indices j in H that correspond to columns with only zeroes as entries. Then B ∪ U contains all column indices in H (Lemma 6.2.3). Moreover, the equation system (Hy = g) fixes each variable y j with j ∈ B to the value t j because H is lower triangular with gaps. Hence,

((AV )y ≤ b) ∪ (Hy = g) is equivalent to A    j∈U    v 1j . . . v nj    • y j    ≤ b -A    j∈B    v 1j . . . v nj    • t j    . (6.1)
Due to Lemma 6.3.4 and 6.2.8, all directions are unbounded in (6.1). This means (6.1) has an integer solution (Lemma 2.8.8) assigning each variable y j with j ∈ U to a t j ∈ Z. (Can be computed via the unit cube test form Chapter 4). We extend this solution to all variables y by setting t j := t j for j ∈ B and we have a mixed solution t ∈ (Q n 1 δ × Z n 1 ) for ((AV )y ≤ b) ∪ (l ≤ Hy ≤ u). Hence, we have via Lemma 6.2.7 a mixed solution s ∈ (Q

n 1 δ × Z n 2 ) for (Ax ≤ b) ∪ (l ≤ Dx ≤ u) with s = V t .
The above proof is constructive and, thereby, also explains how to convert a satisfiable solution (i.e., a certificate of satisfiability) from the bounded part to the complete constraint system. The remainder of the proof of equisatisfiability is trivial:

Corollary 6.3.6 (Double-Bounded Reduction). A split system (Ax ≤ b) ∪ (l ≤ Dx ≤ u) is mixed equisatisfiable to its bounded part (l ≤ Dx ≤ u).

Incremental Implementation

Suppose an SMT theory solver has to solve (Ax ≤ b) ∪ (Dx ≤ c). Moreover, the last problem it has solved was (Ax ≤ b). Then we call the runtime advantage it gains from having already solved a subset of the problem its incremental efficiency. Since all problems sent to an SMT theory solver are incrementally connected, its incremental efficiency is a major factor in determining its total efficiency.

In this section, we explain how to implement our transformation based approach more incrementally efficient and what limits there are with regard to incremental efficiency. We start our discussion with incrementally efficient implementations of the subcomponents of our procedure: finding bounded inequalities and computing the Mixed-Echelon-Hermite normal form.

Finding Bounded Inequalities Incrementally

In Section 6.3, we explained via Lemma 5.5.1 that all bounded directions in Ax ≤ b are equalities in Ax ≤ 0 m and vice versa. Based on this fact, we recommend to use the method outlined in Chapter 5 to compute an equality basis for Ax ≤ 0 m and to determine with it the inequalities in Ax ≤ b that are bounded.

We also recommend the equality basis method because it is incrementally efficient (see Chapter 5.3). This incremental efficiency directly translates to determining bounded inequalities. Since determining the bounded inequalities is the bottleneck of splitting (Section 6.3), the incremental efficiency also translates to splitting a constraint system. Algorithm 19: ExtendI(Hy ≤ u, V, h T m+1 y ≤ b m+1 , j) Input : A system of inequalities Hy ≤ u, where H ∈ Q m×n is in Mixed-Echelon-Hermite normal form and u ∈ Q m δ ; a mixed column transformation matrix V ∈ Q n×n ; an inequality h T m+1 y ≤ b m+1 , where h m+1 ∈ Q n and b m+1 ∈ Q δ ; and a column index j > n 1 such that h m+1j = 0 and h ij = 0 for all i ≤ m Effect : Extends Hy ≤ u by h T m+1 y ≤ b m+1 and transforms it into Mixed-Echelon-Hermite normal form H y ≤ u via mixed column transformation operations, which are stored in V . H has one more non-zero integer column than H and (Hy ≤ u) ⊂ (H y ≤ u ). Output:

(H y ≤ u ,V ) 1 p := PivI(m, H) + 1; 2 H y ≤ u := Insert h T m+1 y ≤ b m+1 between rows p -1 and p of Hy ≤ u; 3 (H , V ) := ReduceLeftI(H , V, p); 4 (H , V ) := ReduceRightI(H , V , p); 5 return (H y ≤ u ,V ); Figure 6.7: ExtendI(Hy ≤ u, V, h T m+1 y ≤ b m+1 , j)
(H y ≤ u ) the inequality h T m+1 y ≤ b m+1 is located in row p. Then ExtendR() swaps column j with column p and uses column operations to eliminate all other coefficients in h T m+1 that fill gaps in H. The result H y ≤ u is then again in MEHNF (and V is the mixed column transformation matrix as specified above). Since all column operations are performed on columns with gaps in H, all inequalities in Hy ≤ u also appear in H y ≤ u , i.e., (Hy ≤ u) ⊂ (H y ≤ u ).

Case 2: h T m+1 fills no rational gap, but an integer gap of H, i.e., there exists a zero column n 1 < j ≤ n in H such that h m+1j = 0. In this case, we have to extend H from p -1 non-zero integer columns to p non-zero integer columns. We do so with the function ExtendI() (Figure 6.7). ExtendI() first inserts the inequality h T m+1 y ≤ b m+1 at an appropriate position p (line 2), to solve the rank requirements we discussed before. So in the new constraint system (H y ≤ u ) the inequality h T m+1 y ≤ b m+1 is located in row p. Then ExtendI() swaps column j with column p and uses column operations to eliminate all other coefficients in h T m+1 that fill gaps in H. The resulting system of inequalities H y ≤ u is then again in MEHNF (and V is the transformation matrix as specified above). Since all column operations are performed on columns with gaps in H, all inequalities in Hy ≤ u also appear in H y ≤ u , i.e., (Hy ≤ u) ⊂ (H y ≤ u ). Figure 6.9: AbstractToInt(H , V , p)

The case distinction over the algorithms ExtendR() and ExtendI() is necessary because of the restrictions we have on our column transformations4 , e.g., we can add multiples of rational columns to integer columns but not vice versa.

Since ExtendR() and ExtendI() change only the new inequality, it holds that (Hy ≤ u) is equivalent to AV y ≤ b. This means that an extended transformation matrix still transforms the previous constraint system into an equisatisfiable MEHNF. We can use this fact to be also decrementally efficient, i.e., to efficiently remove inequalities in the order they were added. In order to remove a T m+1 x ≤ b m+1 from H y ≤ u , we simply remove the last inequality that was added to the constraint system (can be efficiently marked with a flag) to get again (Hy ≤ u). Since (Hy ≤ u) is equivalent to AV y ≤ b, it is not necessary to change the transformation matrix 5 . Thus, we have found an incrementally and decrementally efficient way to compute the MEHNF of a constraint system.

Algorithm 22: ReduceRightI(H , V , p)

Input : A matrix H ∈ Q m+1×n , a mixed column transformation matrix V ∈ Q n×n , and a row and column index p. Effect : Applies mixed column transformations to H until all entries h pi with n 1 < i < p are non-negative and less than h pp . The transformations are also added to V . Output:

(H y ≤ b ,V ) 1 c := lcm{d pj | j ∈ {n 1 + 1, . . . , n} and d pj := denominator of h pj }; 2 s pp := h pp • c; 3 for j ∈ {n 1 + 1, . . . , p -1} do

The Complete Incremental Procedure

Now that we have incrementally efficient subprocedures, we can describe a version of our complete procedure that is incrementally efficient. As a reminder, the non-incremental version of our total procedure works as follows: Our input is a constraint system Ax ≤ b and we want to find a mixed solution for it. To this end, we first compute the equality basis of Ax ≤ 0 m to find the inequalities and directions in Ax ≤ b that are bounded. Next we do a case distinction depending on whether Ax ≤ b is bounded, absolutely unbounded or partially unbounded. If Ax ≤ b is bounded, we find the mixed solution via branch-and-bound6 . If Ax ≤ b is absolutely unbounded, we find the mixed solution via the unit cube test (see Chapter 4). The only slightly complicated case is if Ax ≤ b is partially unbounded. In this case, we first split Ax ≤ b into a split system and transform the double-bounded part into its MEHNF. The double-bounded system in MEHNF is then solved with branch-and-bound.

Now assume that we have done all of the above for Ax ≤ b, but need to incrementally extend it to (Ax ≤ b) ∪ (A x ≤ b ). This means we want to find a mixed solution for (Ax ≤ b) ∪ (A x ≤ b ). If (Ax ≤ b) was already bounded, then we know that (Ax ≤ b) ∪ (A x ≤ b ) will also be bounded and we simply apply branch-and-bound to it. Otherwise, we find the inequalities and directions in (Ax ≤ b) ∪ (A x ≤ b ) that are bounded by extending the equality basis of Ax ≤ 0 m to the equality basis of (Ax ≤ 0 m ) ∪ (A x ≤ 0 m ). In Section 6.4.1, we have shown how to do this incrementally efficient. Next we do a case distinction depending on whether (Ax ≤ b)∪(A x ≤ b ) is bounded, absolutely unbounded or partially unbounded. If (Ax ≤ b) ∪ (A x ≤ b ) is now bounded, we find the mixed solution via branch-and-bound. If (Ax ≤ b)∪(A x ≤ b ) is still absolutely unbounded, we find the mixed solution via the unit cube test (also an incrementally efficient procedure; see Chapter 4). If (Ax ≤ b) ∪ (A x ≤ b ) is still partially unbounded, we continue as follows: We still have the split system for (Ax ≤ b) and can now use our extended equality basis for (Ax ≤ 0 m ) ∪ (A x ≤ 0 m ) to efficiently extend it to a split system for (Ax ≤ b) ∪ (A x ≤ b ). Since adding new inequalities can only add bounded directions, the double-bounded part of the extended split system still contains all bounded inequalities from the previous double-bounded part. This means we can incrementally extend the MEHNF l ≤ Hy ≤ u by the new inequalities in the double-bounded part of (Ax ≤ b) ∪ (A x ≤ b ). In Section 6.4.2, we have shown how to do this incrementally efficient. Finally, we solve the extended double-bounded constraint system (l ≤ Hy ≤ u) ∪ (l ≤ H y ≤ u ) with branch-and-bound. Since we only add inequalities to the running constraint system (l ≤ Hy ≤ u), we can continue our branch-and-bound search incrementally efficient. This shows that most parts of our procedure can be implemented incrementally efficient. However, there are two limits to the incremental efficiency. First of all, we have to store multiple constraint systems in our memory to stay incrementally efficient: we need one system to store the current equality basis, so we can later extend it; we need one system to store the current MEHNF, so we can later extend it; we need the current transformation matrix of the MEHNF transformation, so we can later extend it; and we need one copy of the MEHNF to perform branch-and-bound on. Secondly, we do not know how to make the assignment/solution conversion incrementally efficient, i.e., how to convert the mixed solution of the transformed system to a mixed solution of the original system in an incrementally efficient way (see Lemma 6.3.5 for the non-incremental subprocedure). However, this second limitation is in reality not a problem because there are ways to avoid the conversion until we know that the complete problem is satisfiable. So the conversion is used at most once for each SMT input problem. In the next subsection, we will elaborate why this is the case.

Avoiding the Solution Conversion

In order to explain why we can avoid the solution conversion, we first have to distinguish the origin of the incrementally connected problems, i.e., the origin of the problems sent from the SMT solver to the SMT theory solver. There are typically two reasons a theory solver might receive incrementally connected problems from the SMT solver:

(1) The SMT solver tries to prune some partial models (i.e., conjunctions of literals) that are theory unsatisfiable. This case is actually not necessary for a complete SMT solver 7 . It is just a trick to speed-up the boolean search of the SMT solver. In fact, it is already too expensive for the theory solver to check all partial models. Instead, theory solvers typically just check partial models when the SAT solver is about to do a decision. And even then the check is often just a sound approximation of the complete theory solver because the complete check is too expensive for some theories. One of those theories is in fact linear mixed/integer arithmetic. For this theory, most SMT solvers check only the rational relaxation of the partial models for theory satisfiability. So this source of incrementally connected problems is not relevant to our complete approach.

(2) The SMT solver combines multiple theory solvers via the Nelson-Oppen method. As part of the Nelson-Oppen method, (2.1) each theory solver has to first determine the satisfiability of their own conjunctions of literals. (2.2) Then the theory solvers incrementally send to each other (negated) equalities over constant function symbols and test these extended problems for satisfiability. (2.3) This continues until they find a complete and satisfiable equivalence class over the constant function symbols. All of the above can be done with our transformation scheme without converting the intermediate solutions to the original system. Unfortunately, most SMT solvers rely on the intermediate solutions to the original system to guess the (negated) equalities they send in step (2.2). At a first glance, case (2) seems like it actually needs the solution conversion via Lemma 6.3.5. However, there is an easy and reasonable way to avoid it. Instead of using the intermediate solution to the complete original system, we just use the intermediate solution to the double-bounded part of the original system. This solution can be efficiently computed with the transformation matrix V , i.e., x := V y is the solution to the double-bounded part of the original system if y is the solution to the transformed system. This is a reasonable approximation for the guesses in (2.2) because we know that the unbounded part is irrelevant to the satisfiability of the original system (Corollary 6.3.6).

We conclude that our total incremental procedure never has to convert a complete solution more than once. So the procedure should be incrementally efficient in practice. However, we are unable to test this claim with experiments since we do not have a SMT solver that supports other theories besides linear arithmetic.

Experiments

We integrated the Mixed-Echelon-Hermite transformation and the Double-Bounded reduction into our own theory solver SPASS-IQ v0.3 and ran it on four families of newly constructed benchmarks. 8 Once with the transformations turned on (SPASS-IQ) and once with the transformations turned off (SPASS-IQ-Off ). If SPASS-IQ encounters a system Ax ≤ b that is not explicitly bounded, i.e., where not all variables have an explicit upper and lower bound, then it computes an equality basis for Ax ≤ 0 m (see Chapter 5). This basis is used to determine whether the system is implicitly bounded, absolutely unbounded or partially bounded, as well as which of the inequalities are bounded. Our solver only applies our two transformations if the problem is partially unbounded. The resulting equisatisfiable but bounded problem is then solved via branch-and-bound. The other two cases, absolutely unbounded and implicitly bounded, are solved respectively via the unit cube test (see Chapter 4) and branch-and-bound on the original system. Our solver also converts any mixed solutions from the transformed system into mixed solutions for the original system following the proof of Lemma 6.3.5. Rational conflicts are converted between the two systems by using Corollary 6.2.9.

To evaluate the efficiency of our transformations, we compared SPASS-IQ with several other solvers for systems of linear inequalities. The focus of our experiments are partially unbounded problems since SPASS-IQ applies our transformations only on this class of problems. However, before our first publication on the Mixed-Echelon-Hermite transformation and the Double-Bounded reduction, the SMT-LIB (Satisfiability Modulo Theories Library) benchmarks for QF LIA (quantifier-free linear integer arithmetic) [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF] contained only a few partially unbounded problems: two in arctic-matrix, one in CAV-2009, five in cut lemmas, three in slacks, and four in tropical-matrix. Since there were only so few partially unbounded problems in the SMT-LIB, we created in addition four new benchmark families:9 horizontal axis: # of solved instances; vertical axis: time (seconds) Figure 6.12: SPASS-IQ compared to various SMT solvers on the RandomUnbd and FlippedRandomUnbd benchmark families SlackedQFLIA: are linear integer benchmarks based on the SMT-LIB classes CAV-2009 [START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF], cut lemmas [START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF], and dillig [START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF]. We simply took all of the unsatisfiable benchmarks and replaced in them all variables x with x +x -where x + and x -are two new variables such that x + , x -≥ 0. This transformation, called slacking, is equisatisfiable and the slacked version of the dillig benchmarks, called slacked [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], is already in the SMT-LIB. Slacking turns any unsatisfiable problem into a partially unbounded one. Hence, all problems in SlackedQFLIA are partially unbounded. Slacking is commonly used to integrate absolute values into linear systems or for solvers that require non-negative variables [START_REF] Schrijver | Theory of linear and integer programming[END_REF].

RandomUnbd: are linear integer benchmarks that are all partially unbounded and satisfiable with 10, 25, 50, 75, and 100 variables. All problems in this benchmark family are randomly created via a sagemath script.

FlippedQFLIA and FlippedRandomUnbd: are linear mixed benchmarks that are all partially unbounded. They are based on SlackedQFLIA and RandomUnbd. We constructed them by first copying ten versions of the integer benchmarks and then randomly flipping the type of some of the variables to rational (probability of 20%). Some of the flipped instances of SlackedQFLIA became satisfiable.

For the experiments, we used a Debian Linux cluster and allotted to each problem and solver combination 1 core of an Intel Xeon E5620 (2.4 GHz) processor, 4 GB RAM, and 40 minutes. The plots in Figures 6.11, 6.12, and 6.13 depict the results of the different solvers. In the legends of the plots, the numbers behind the solver names are the number of solved instances. For FlippedQFLIA, there are two numbers to indicate the number of satisfiable/unsatisfiable instances solved. This is only necessary for FlippedQFLIA because it is the only tested benchmark family with satisfiable and unsatisfiable instances. (We verified that the results match if two solvers solved the same problem.)

Comparison with SMT Solvers

We compared our solver, SPASS-IQ, with some of the state-of-the-art SMT solvers currently available for linear arithmetic: CVC4 (v1.6) [START_REF] Barrett | CVC4. In CAV[END_REF], MathSAT5 (v5.5.2) [START_REF] Cimatti | The MathSAT5 SMT solver[END_REF], Princess (version from 2018-10-26) [START_REF] Rümmer | A constraint sequent calculus for first-order logic with linear integer arithmetic[END_REF], SMTInterpol (v2. [START_REF] Althaus | Superposition modulo linear arithmetic SUP(LA)[END_REF][START_REF] Baaz | Normal form transformations[END_REF][START_REF] Bachmair | On restrictions of ordered paramodulation with simplification[END_REF][START_REF] Bareiss | Sylvester's identity and multistep integer-preserving Gaussian elimination[END_REF][START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF][START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF][START_REF] Barrett | The SMT-LIB Standard: Version 2.0[END_REF][START_REF] Barrett | Satisfiability Modulo Theories[END_REF][START_REF] Baumgartner | Hierarchic superposition with weak abstraction[END_REF][START_REF] Beale | An alternative method for linear programming[END_REF][START_REF] Berman | Precise bounds for Presburger arithmetic and the reals with addition: Preliminary report[END_REF][START_REF] Berman | The complexity of logical theories[END_REF][START_REF] Bertacco | A feasibility pump heuristic for general mixed-integer problems[END_REF][START_REF] Bézout | Théorie générale des équations algébriques[END_REF] [START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF], Yices (v2.6.0) [START_REF] Dutertre | Yices 2.2[END_REF], and Z3-4.6.0 (v4.8.1) [START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF]. (See Figures 6.11 and 6.12 for the results.)

Most of these solvers employ a branch-and-bound approach with an underlying dual simplex solver [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF], which is also the basis for our own solver. (As far as we are aware, our own solver, SPASS-IQ, is the only branch-andbound based solver that employs a technique that guarantees termination.) We are, however, particularly interested in the three solvers not based on this approach: Ctrl-Ergo, Princess, and SMTInterpol.

Ctrl-Ergo uses an approach that is complete over linear integer arithmetic but cannot handle linear mixed arithmetic [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF]. The approach works by dynamically eliminating one linear independent bounded direction at a time via transformation. The disadvantages of the dynamic approach are that it is very restrictive and does not leave enough freedom to change strategies or to add complementing techniques. Moreover, Ctrl-Ergo uses this transformation approach for all problems and not only the partially unbounded ones, which sometimes leads to a massive overhead on bounded problems.

Princess uses a constraint sequent calculus that is not only complete for linear integer arithmetic but also for quantified linear integer arithmetic. It terminates because it simulates in the worst case the complete and terminating Omega Test [START_REF] Pugh | The omega test: a fast and practical integer programming algorithm for dependence analysis[END_REF]. The broader focus of the constraint sequent calculus also has one big disadvantage: it is less efficient than more specialized methods when it only has to determine satisfiability of a problem. This is especially true for determining satisfiability of (partially) unbounded problems, which is confirmed by our experiments. Moreover, the constraint sequent calculus cannot handle linear mixed arithmetic.

SMTInterpol extends branch-and-bound via the cuts from proofs approach, which uses the Mixed-Echelon-Hermite transformation to find more versatile branches and cuts [START_REF] Christ | Cutting the mix[END_REF]. Although the procedure is not complete, the similarities to our own approach make an interesting comparison. Actually, the Double-Bounded reduction alone would be sufficient to make SMTInterpol terminating since it already builds branches via a Mixed-Echelon-Hermite transformation.

Although our solver could not solve all problems (due to time and memory limits) it was still able to solve more problems than the other solvers. It was also faster on most instances than the other solvers. On some of the unsatisfiable, partially unbounded benchmarks, Ctrl-Ergo is better than SPASS-IQ. This is due to its conflict focused, dynamic approach. For the same reason, Ctrl-Ergo is slower on the satisfiable, partially unbounded benchmarks. Only SPASS-IQ, Ctrl-Ergo, and Yices solved all of the fifteen original SMT-LIB benchmarks that are partially unbounded, though the complete methods were still a lot faster (SPASS-IQ took 23s, Ctrl-Ergo took 42s, and Yices took 1273s). On one of these benchmarks, 20-14.slacks.smt2 from slacks, all other solvers seem to diverge. Another interesting result of our experiments is that relaxing some integer variables to rational variables seems to make the problems harder for SMT solvers instead of easier. We expected this for our transformations because the resulting systems become more complex and less sparse, but it is also true for the other solvers. The reason might be that bound refinement, a technique used in most branch-and-bound implementations (see also Chapter 2.7.4), is less effective on mixed problems.

Comparison with MILP Solvers

We compared our solver with several solvers for mixed-integer programming (MILP) (see Figure 6.13): the two non-commercial solvers GLPK (v4.65) [START_REF] Makhorin | GNU Linear Programming Kit (GLPK)[END_REF] and SCIP (v6.0.0) [START_REF] Gleixner | The SCIP Optimization Suite 6.0[END_REF] as well as the commercial solver Gurobi (v7.52) [START_REF]Gurobi Optimizer Reference Manual[END_REF]. For these experiments, we used the same benchmarksalthough converted into the MPS (Mathematical Programming System) format-and the same experiment parameters as for our experiments with the SMT solvers. In General, mixed-integer programming solvers have an advantage over standard SMT theory solvers because (i) they are not required to be exact and sound, which allows them to use floating-point arithmetic, and (ii) they are not required to be incrementally efficient, which means they can use much more elaborate techniques. (As far as we are aware, none of the MILP solvers uses a technique that guarantees termination.) Despite these advantages, SPASS-IQ is faster and solves more problems from the RandomUnbd benchmarks than GLPK, Gurobi, and SCIP. On the SlackedQFLIA benchmarks, SPASS-IQ is again faster and solves more problems than GLPK and SCIP, but is less efficient than Gurobi. Gurobi is more efficient on the SlackedQFLIA benchmarks because it uses a preprocessing technique that undoes the slacking transformation. This means Gurobi solves the original bounded versions of the SlackedQFLIA benchmarks, which are by far easier. If we turn this preprocessing technique off (see Gurobi-Off), then SPASS-IQ is also faster and solves more problems than Gurobi.

We also tested the MILP solvers on the linear mixed arithmetic benchmark families FlippedQFLIA and FlippedRandomUnbd. At a first glance, GLPK, Gurobi, and SCIP seemed to improve compared to their results on the integer versions of the benchmarks. However, this improvement was only possible due to the unsoundness of the three solvers. We determined that GLPK returned the wrong result on at least 72 problems, Gurobi returned the wrong result on at least 320 problems, and SCIP returned the wrong result on at least 249 problems. Since GLPK, Gurobi, and SCIP were so unreliable on the linear mixed arithmetic problems, no fair comparison with SPASS-IQ was possible.

Further Remarks on SPASS-IQ

The time SPASS-IQ needs to detect the bounded inequalities and to apply our transformations is negligible. This is even true for the implicitly bounded problems we tested. As mentioned before, we do not have to apply our transformations to terminate on bounded problems. This is also the only advantage we gain from detecting that a problem is implicitly bounded. Since there is no noticeable difference in the runtime, we do not further elaborate the results on bounded problems, e.g. with graphs. An actual disadvantage of our approach is that the Mixed-Echelon-Hermite transformation almost always increases the density of the coefficient matrix as well as the absolute size of the coefficients. Both are important factors for the efficiency of the underlying simplex solver.

Summary

We have presented the Double-Bounded reduction (Lemma 6.3.2 & Corollary 6.3.6) and the Mixed-Echelon-Hermite transformation (Lemma 6.2.11). We have shown that both transformations together turn any constraint system into a mixed equisatisfiable system that is also bounded (Lemma 6.2.3). This is sufficient to make branch-and-bound, and many other linear mixed decision procedures, complete and terminating. We have also shown how to convert certificates of rational (un)satisfiability efficiently between the transformed and original systems (Corollary 6.2.9 & Lemma 6.3.5). Moreover, experimental results on partially unbounded benchmarks [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF] show that our approach is also efficient in practice. Our approach can be nicely combined with the extensive branch-and-bound framework and its many extensions, where other complete techniques cannot be used in a modular way [START_REF] Bobot | A simplex-based extension of Fourier-Motzkin for solving linear integer arithmetic[END_REF][START_REF] Bromberger | Linear integer arithmetic revisited[END_REF].

Chapter 7

Implementation of SPASS-IQ

We have presented in the previous chapters several benchmark evaluations for our decision procedures. These evaluations are mostly based on our linear arithmetic theory solver SPASS-IQ. Moreover, they all focus on one single technique relevant to the respective chapter, but never on their combination in one system. In this chapter, we remedy this negligence and describe and evaluate the overall implementation of SPASS-IQ. SPASS-IQ is a theory solver for linear arithmetic. Its input problems are finite sets of inequalities containing arithmetic variables, which is equivalent to our definition of standard input problems from Chapter 2.4. Given such an input problem, SPASS-IQ's goal is to either (i) find an assignment for the variables that satisfies all of the input inequalities and the variable types (integer/rational), or (ii) to determine that no such assignment exists.

The branch-and-bound implementation is more of a supervisor for the simplex implementation. We say so because the branch-and-bound implementation may coordinate the search for a mixed or integer solution, but the majority of the actual search/calculation is still done by the simplex implementation. To be more precise, the branch-and-bound implementation mostly evaluates the assignments and conflicts returned by the simplex implementation and based on this evaluation provides the simplex implementation with new subproblems to solve. This means that the efficiency of our branch-and-bound implementation highly depends of the efficiency of our simplex implementation.

By far not all techniques implemented in SPASS-IQ are also unique features of SPASS-IQ. The techniques that appeared first in SPASS-IQ are the unit cube test (Chapter 4) and the bounding transformations (Chapter 6). Further important techniques implemented in SPASS-IQ have already been available in other SMT (theory) solvers such as CVC4 [START_REF] Barrett | CVC4. In CAV[END_REF], MathSAT [START_REF] Cimatti | The MathSAT5 SMT solver[END_REF], Yices [START_REF] Dutertre | Yices 2.2[END_REF], and Z3 [START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF], but not all in one tool: the implementation of branchand-bound as a separate theory solver and a number of improvements to the simplex implementation such as a priority queue for pivot selection, integer coefficients instead of rational coefficients, dynamically switching between native and arbitrary precision integers, and backing-up versus recalculating simplex states. Although these techniques are contained in existing SMT theory solvers, not all have been described in the respective literature.

Experimental Setup

For this chapter, we have performed several experiments in order to estimate the impact of SPASS-IQ's various features. However, it is hard to estimate the impact of a theory solver (e.g., incremental efficiency and conflict generation) that is intended to be a module of a more complex solver (e.g., a CDCL(LA) solver) if we test it outside of that complex solver. Therefore, we do not test SPASS-IQ directly, but SPASS-SATT, the CDCL(LA) extension of SPASS-IQ. (For more details on SPASS-SATT see Chapter 8.)

The benchmarks for our experiments are the 1649 SMT-LIB benchmarks for quantifier free linear rational arithmetic (QF LRA) and the 6947 SMT-LIB benchmarks for quantifier free linear integer arithmetic (QF LIA) from the SMT-LIB (satisfiability modulo theories library) [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF]. All experiments compare different configurations of SPASS-SATT with each other. The experiments are preformed on a Debian Linux cluster from which we allotted one core of an Intel Xeon E5620 (2.4 GHz) processor, 8 GB RAM, and 40 minutes to each combination of problem and SPASS-SATT configuration.

Simplex Implementation

The basis of SPASS-IQ is an implementation of the simplex algorithm, which we explained in detail in Chapter 2.7. The general efficiency of the simplex algorithm is heavily influenced by the data structures that we use in our implementation. An implementation with optimal data structures can be several orders of magnitude faster than a naive implementation. This section, therefore, includes descriptions on the data structures we use to represent variables, bounds, and the simplex tableau (Subsection 7.1.4). In addition, we present a data structure that efficiently selects violated basic variables for pivoting (Subsection 7.1

.2).

There also exist other design choices, besides mere data structures, that have a heavy influence on the efficiency of our implementation. For instance, the pivoting strategy. In Subsection 7.1.1, we not only present the pivoting strategy used by SPASS-IQ, but we also compare it with several other pivoting strategies from various SMT solvers. Moreover, we describe an alternative backtracking technique (Subsection 7.1.3) to the backup technique we have presented in Chapter 2.7. SPASS-SATT, the CDCL(LA) solver based on SPASS-IQ, solves more instances from the SMT-LIB benchmarks with this alternative technique backtracking than with the original backtracking technique.

Pivoting

The simplex algorithm pivots variables in every iteration. Therefore, pivoting is one of the (if not the) most important operation(s) for the simplex algorithm. Pivoting itself is a substitution that turns a basic variable y i into a non-basic variable and, in exchange, a non-basic variable z j into a basic variable. For this reason, we also call the two variables y i and z j the pivoting variables.

The pivots done by the simplex algorithm are not absolutely arbitrary, i.e., the pivoting variables actually have to fulfill certain conditions to be selected for pivoting (see also Check() in Figure 2.2). These conditions are as follows:

The simplex algorithm first selects the basic variable y i for pivoting. Any basic variable y i is a potential candidate for pivoting as long as it is violated , i.e., the current assignment β(y i ) for the variable y i violates either the upper bound (i.e., β(y i ) > U(y i )) or the lower bound (i.e., β(y i ) < L(y i )). After the basic variable y i is selected for pivoting, the non-basic variable z j is selected. Any non-basic variable z j is a potential candidate for pivoting as long as it fulfills the following two conditions: (i) the variable z j has a non-zero coefficient a ij in the row of the selected basic variable y i and (ii) there is no tight bound in the variables changing direction, i.e., β(z j ) < U(z j ) if the changing direction is upper and β(z j ) > L(z j ) if the changing direction is lower.

The last point is very vague so let us explain it in more detail. If a bound of a basic variable y i is violated, then we need to change the variable's assigned value β(y i ) so the bound is no longer violated. However, we cannot change the assignment of a basic variable directly because the assigned value β(y i ) is defined through the tableau and the values assigned to the non-basic variables. This means we have to change the values assigned to the non-basic variables in order to fix the violated basic variable.

If the lower bound of the selected basic variable y i is violated, then we need to increase the value β(y i ) assigned to y i . This is possible by either increasing the value β(z j ) assigned to a non-basic variable z j with a positive coefficient in y i 's row (a ij > 0) or by decreasing the value β(z j ) assigned to a non-basic variable z j with a negative coefficient in y i 's row (a ij < 0). Symmetrically, we need to somehow decrease the value β(y i ) assigned to y i if the upper bound of the selected basic variable y i is violated. Again, this is possible by either decreasing the value β(z j ) assigned to a non-basic variable z j with a positive coefficient in y i 's row (a ij > 0) or by increasing the value β(z j ) assigned to a non-basic variable z j with a negative coefficient in y i 's row (a ij < 0).

Based on this change in assignment, we define the changing direction of a non-basic variable. The changing direction of a non-basic variable z j is "upper" if the value β(z j ) needs to be increased to fix the assignment for y i . And the changing direction of a non-basic variable z j is "lower" if the value β(z j ) needs to be decreased to fix the assignment for y i . This means that the second non-basic pivoting condition-there is no tight bound in the variables changing direction-prevents us from selecting a non-basic variable that cannot be changed without violating one of its own bounds.

Pivoting Strategies

The pivoting conditions of the simplex algorithm are necessary to prevent the simplex algorithm from repeatedly pivoting the same two variables, which would cause divergence. Nonetheless, the conditions are still too weak to always guarantee exactly one candidate pair of pivoting variables and to prevent all types of divergence. Therefore, we need a pivoting strategy, i.e., a strategy for selecting the pivoting variables, that resolves the nondeterminism and prevents the divergence.

The simplex algorithm can only diverge if it runs into a pivoting cycle, i.e., a series of pivots that leads back to the same set of non-basic variables and the same assignment. The pivoting conditions alone are enough to prevent all pivoting cycles consisting of upto two pivots. We need, however, a terminating pivoting strategy, e.g., Bland's rule [START_REF] Robert | New finite pivoting rules for the simplex method[END_REF], to avoid all other cycles.

The terminating pivoting strategies have the disadvantage that they typically need more pivots to reach a solution if there are no cycles than greedy but potentially diverging strategies. We, therefore, have to find a compromise between termination and efficiency in practice. Our solution is to first do a finite number of pivots according to a greedy but potentially diverging strategy and afterwards continue with a terminating pivoting strategy. 1 This combination of greedy and terminating pivoting strategy is also commonly used by other SMT theory solvers, e.g., in CVC4 [START_REF] Barrett | CVC4. In CAV[END_REF], SMTInterpol [START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF], veriT [START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF], Yices [START_REF] Dutertre | Yices 2.2[END_REF], and Z3 [START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF].

We have actually implemented several pivoting strategies inside SPASS-IQ (command line option -LAPR <id>) and compared. The most promising strategies are listed below. Most of them rely on a strict total variable order ≺.

Bland's rule (-LAPR 1): Basic variable strategy: select the smallest candidate variable according to ≺. Non-basic variable strategy: select the smallest candidate variable according to ≺.

CVC4's rule [START_REF] Barrett | CVC4. In CAV[END_REF] (-LAPR 2):2 basic variable strategy: primarily prefer candidate variables with minimum error, i.e., variables y i with minimum absolute difference |b i -β(y i )| between the violated bound value b i and the assigned value β(y i ); then prefer the smallest candidate variable according to ≺. non-basic variable strategy: primarily prefer candidate variables without any bounds; then prefer candidate variables with the least number of non-zero coefficients; finally prefer the smallest candidate variable according to ≺. additional comments: switches after a finite number of pivoting steps to Bland's rule.

SMTInterpol's rule [START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF] (-LAPR 3): Basic variable strategy: primarily prefer candidate variables with the least number of non-zero coefficients in their tableau row; then prefer the smallest candidate variable according to ≺. non-basic variable strategy: primarily prefer candidate variables without any bounds; then prefer candidate variables without a bound in their changing direction; then prefer candidate variables with the least number of non-zero coefficients; finally prefer the smallest candidate variable according to ≺. additional comments: switches after a finite number of pivoting steps to Bland's rule.

Maximum error rule [START_REF] Barrett | CVC4. In CAV[END_REF] (-LAPR 4):3 basic variable strategy: primarily prefer candidate variables with maximum error, i.e., variables y i with maximum absolute difference |b i -β(y i )| between the violated bound value b i and the assigned value β(y i ); then prefer the smallest candidate variable according to ≺. non-basic variable strategy: primarily prefer candidate variables without any bounds; then prefer candidate variables with the least number of non-zero coefficients; finally prefer the smallest candidate variable according to ≺. additional comments: switches after a finite number of pivoting steps to Bland's rule. The maximum error rule is not used by any SMT solver as far as we are aware.

Prefer unbounded rule (-LAPR 5): basic variable strategy: primarily prefer candidate variables that have nonbasic variables in their row with an unbounded changing direction; then prefer the smallest candidate variable according to ≺. non-basic variable strategy: primarily prefer candidate variables without any bounds; then prefer candidate variables without a bound in their changing direction; then prefer candidate variables with the least number of non-zero coefficients; finally prefer the smallest candidate variable according to ≺. additional comments: switches after a finite number of pivoting steps to Bland's rule. The prefer unbounded rule is not used by any SMT solver as far as we are aware.

SPASS-SATT's rule (-LAPR 0): basic variable strategy: select the smallest candidate variable according to ≺. non-basic variable strategy: primarily prefer candidate variables without any bounds; then prefer candidate variables with the least number of non-zero coefficients; finally prefer the smallest candidate variable according to ≺. additional comments: switches after a finite number of pivoting steps to Bland's rule.

We implemented these strategies into SPASS-IQ and compared their overall efficiency through benchmark experiments with SPASS-SATT, the CDCL(LA) extension of SPASS-IQ. We present the benchmark results in the cactus plots in Figures 7.1-7.4. In the legends of the plots, the two numbers in brackets behind each pivoting strategy name are the numbers of satisfiable and unsatisfiable instances that we can solve with the respective strategy.

From the cactus plots, we can see that Bland's rule alone performs much worse than most of the other pivoting strategies. Moreover, the pivoting strategies that do not switch to Bland's rule at some point perform worse than their counterparts that do switch to Bland's rule. This supports our initial assumption that we need to combine a greedy pivoting strategy with a terminating pivoting strategy in order to receive a generally efficient pivoting strategy. The best pivoting strategies for QF LRA and QF LIA are SMTInterpol's rule and SPASS-SATT's rule, respectively. This is also the reason why we selected SMTInterpol's rule and SPASS-SATT's rule as SPASS-IQ's pivoting strategies for QF LRA and QF LIA, respectively. However, we do want to remark that the difference between the best strategies and their runner-ups is so small that they could be attributed to performance fluctuations of our cluster. Therefore, the runner-up pivoting strategies are most likely equally good choices.

Violated Variable Heap

The Check() function selects in every iteration of its while-loop a violated variable, i.e., a basic variable with a violated bound (see Figure 2.2 line 2). Typically, we have a large number of basic variables, but only a small subset of them is actually violated in a given iteration of the simplex algorithm. Our implementation goal is, therefore, to reduce the number of basic variables visited in each iteration and, if possible, reduce it even to the subset of actually violated variables.

We have decided to implement our variable selection through a violated variable heap (VV heap). (Similar heaps can be found in all of the state-ofthe-art theory solvers for SMT [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].)

The VV heap contains exactly all currently violated variables during variable selection. Inside the heap, the variables are ordered according to their selection preference. The heap is, therefore, mostly a priority queue and its top element is the violated variable we want to select according to our selection preference. However, our heap has one big difference compared to other priority queues: it alternates between two phases. The first phase is called the assertion or buffering phase. Our heap is in the assertion phase while we are making a series of assertions. During this phase, our heap is empty and we do not actually communicate with it directly. Instead, we fill a buffer with all basic variables that change their bounds or their assignment during the assertions.

The second phase is called the check or pivoting phase. Our heap is in the check phase whenever we are inside a Check() call. At the beginning of the check phase, we fill the heap with the basic variables in our buffer that are actually violated. This means that the top element on the heap is also the violated variable we want to select according to our selection preference. We extract this variable from the heap, pivotAndUpdate() with it, and inform the heap of all basic variables that have changed their assignment during the pivot. Based on this information, we are able to add all newly violated variables to the heap and remove all variables from the heap that are no longer violated. This means our heap contains again exactly all currently violated variables.

We continue selecting violated variables in this way until we have found a conflict or there are no more violated variables. This variable selection implementation is more efficient than the naive implementation because it only visits a basic variable when it actually changes in some way.

Greedy Conflict Detection and Returning Multiple Conflicts

There exists one extension that should improve almost all selection preferences: prefer a violated variable if it also describes a conflict. We call this extension greedy conflict detection and it should be beneficial because Check() stops as soon as it detects a conflict.

However, we do not extend all of our selection preferences explicitly with greedy conflict detection. Instead, we added greedy conflict detection directly into our VV heap implementation. This makes sense because we already inform the VV heap of all basic variable changes and, therefore, of all changes that could turn a basic variable into a conflict.

Adding greedy conflict detection to the VV heap also has another advantage. If we implement it in this way, then we detect not only one conflict variable but all conflict variables. This means that we can return multiple conflicts at once without any overhead. 4 (Similar greedy and multiple conflicts schemes can be found in other theory solvers for SMT, e.g., in CVC4 [START_REF] Barrett | CVC4. In CAV[END_REF].)

In spite of these arguments, we were unable to confirm a guaranteed benefit from either greedy conflict detection or learning multiple conflicts in our benchmark experiments. We present the benchmark results in the four scatter plots in . All figures compare two versions of SPASS-SATT, the CDCL(LA) extension of SPASS-IQ. Figure 7.5 examines the effect of greedy conflict detection on the QF LRA benchmarks. To this end, we compare the default version of SPASS-SATT with greedy conflict detection turned on (horizontal axis; command-line option -LAGC 1) with the version of SPASS-SATT that does not use greedy conflict detection (vertical axis; command-line option -LAGC 0). Symmetrically, Figure 7.6 examines the effect of greedy conflict detection on the QF LIA benchmarks with the same solver configurations. Figure 7.7 examines the effect of learning (if possible) multiple conflicts after each unsatisfiable Check() call on the QF LRA benchmarks. To this end, we compare the default version of SPASS-SATT that can produce multiple conflicts at once (horizontal axis; command-line option -LAMC 1) with the version of SPASS-SATT that does not (vertical axis; command-line option -LAMC 0). Symmetrically, Figure 7.8 examines the effect of learning (if possible) multiple conflicts after each unsatisfiable Check() call on the QF LIA benchmarks with the same solver configurations.

In all four of our experiments, the default version of SPASS-SATT, which employs greedy conflict detection and learns (if possible) multiple conflicts, solves slightly more problem instances. However, the increase in solved instances happens so close to the timeout that it could be attributed to performance fluctuations of our cluster. Apart from that, the figures show that 10 -2 10 -1 10 0 Greedy Runtime (seconds) Lazy Runtime (seconds) QF LRA Greedy vs. Lazy Conflict Detection SPASS-SATT with greedy conflict detection solves 1607 instances SPASS-SATT with lazy conflict detection solves 1604 instances Figure 7.5: Greedy vs. lazy conflict detection on the QF LRA SMT-LIB benchmarks there are roughly as many problem instances where greedy conflict detection helps as there are problem instances where it hinders SPASS-SATT's search. We assume that the desired benefit of greedy conflict detection, i.e., ending Check() earlier, is so minor that it is easily outweighed by the unpredictable changes it causes in the remainder of the search. The same can be said about learning multiple conflicts. The smoother continuation point has the result that SPASS-IQ with the HardBacktrack() function is on average more efficient than SPASS-IQ with the standard Backtrack() function. Again, we support this argument by performing benchmark experiments with SPASS-SATT. We present the benchmark results through two scatter plots (Figures 7.10 & 7.11). Figure 7.10 compares the two backtrack functions on the QF LRA benchmarks. To this end, we compare the default version of SPASS-SATT, i.e., SPASS-SATT with HardBacktrack() as its backtrack function (horizontal axis; commandline option -b 0), with the version of SPASS-SATT that uses Backtrack() as its backtrack function (vertical axis; command-line option -b 1). Symmetrically, Figure 7.11 compares the two backtrack functions on the QF LIA benchmarks with the same solver configurations.

As mentioned before, SPASS-SATT with HardBacktrack() performs on average better than SPASS-SATT with Backtrack(). In the QF LIA benchmarks, there even exists one benchmark family, viz., the convert benchmarks, where SPASS-SATT with HardBacktrack() solves much more problems than with Backtrack() (see Figure 7.12). In all other benchmark families, there are roughly as many problem instances where HardBacktrack() helps as there are problem instances where it hinders SPASS-SATT's search. The latter is most likely caused by benchmark instances where the backup solution is already a good continuation point and the recalculation is therefore just more expensive. Moreover, the increase in solved instances from the QF LRA benchmarks could be attributed to performance fluctuations of our cluster because those instance are solved so close to the timeout.

Data Structures

The most basic data structures in our linear arithmetic solver are based on the simplex implementation. They represent numbers, variables, bounds, and simplex tableaux. However, they are not just used in the simplex implementation but in fact in the whole implementation because they embody the most basic constructs of linear arithmetic, i.e., the variables that we need to assign and the constraints that define our satisfiable assignments, but which we transformed into bounds and tableau rows.

Multiprecision Arithmetic

SPASS-IQ is an arithmetic theory solver. This means that we have to represent numbers internally, which might sound easier than it actually is. First of all, SPASS-IQ has to handle three types of numbers: integers Z, rationals Q, and δ-rationals Q δ .

The first two types are generally very popular and there already exist several libraries that contain efficient representations for them [START_REF] Granlund | GNU MP: The GNU Multiple Precision Arithmetic Library[END_REF][START_REF] Hart | FLINT: Fast Library for Number Theory[END_REF]. The most obvious one are the integer and float types that are native to the C language. This representation is very efficient because their operations are optimized on the hardware level. This is also why we call them hardware integers and floats. They are, however, not arbitrarily precise, e.g., hardware integers have a finite range and hardware floats have to round numbers to reach a higher range than integers. This is problematic because SMT solvers-and, therefore, theory solvers-were originally designed for various verification tasks, e.g., hardware verification, software verification, and theorem proving. These task require that our implementation is sound, which means that we have to avoid rounding (errors), and, if possible, complete, which means that our numbers cannot be restricted to a finite range. Therefore, hardware integers and floats are not a good choice for our implementation.

Instead of hardware arithmetic, we are using multiple/arbitrary precision arithmetic libraries to represent our integers and rationals [START_REF] Granlund | GNU MP: The GNU Multiple Precision Arithmetic Library[END_REF][START_REF] Hart | FLINT: Fast Library for Number Theory[END_REF]. 5 Multiple/arbitrary precision arithmetic libraries typically represent integers and rational numbers through variable-length arrays of hardware integers. Therefore, their range is only limited by the available memory of the executing machine. Their operations stay efficient both for big and small numbers because they are using state-of-the-art algorithms written in highly optimized assembly code. FLINT Runtime (seconds) GMP Runtime (seconds) QF LRA FLINT vs. GMP SPASS-SATT with FLINT solves 1607 instances SPASS-SATT with GMP solves 1604 instances Figure 7.13: FLINT vs. GMP on the QF LRA SMT-LIB benchmarks SPASS-IQ can be compiled with one of two arbitrary precision arithmetic libraries: the GNU Multiple Precision Library (GMP) [START_REF] Granlund | GNU MP: The GNU Multiple Precision Arithmetic Library[END_REF] and the Fast Library for Number Theory (FLINT) [START_REF] Hart | FLINT: Fast Library for Number Theory[END_REF][START_REF] Hart | Fast library for number theory: An introduction[END_REF]. Both libraries have their own advantages and disadvantages. For instance, GMP is mainly focused on representing big numbers and, therefore, on efficient algorithms for big number arithmetic. GMP is, however, much slower on small numbers than the native hardware types. This is problematic because verification problems might break the limits of hardware types, but rarely do so.

In contrast, FLINT's performance on small numbers comes much closer to the performance of hardware arithmetic. This is possible because FLINT is able to dynamically switch between hardware types and arbitrary precision types. Otherwise, FLINT is mostly an extension of GMP. This means that FLINT performs similarly to GMP on big numbers, with the exception of the overhead caused by the checks for the dynamic switch to hardware types. Therefore, FLINT is faster on problems with mostly small numbers and GMP is faster on problems with mostly big numbers. We personally prefer FLINT because we typically have to deal with more small numbers than big numbers. For a more detailed comparison of the performance of GMP and FLINT see Figures 7.28 & 7.14. In these figures we compare two versions of SPASS-SATT over the SMT-LIB benchmarks [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF]. The first version uses GMP (vertical axis) the second version FLINT (horizontal axis). Clearly, SPASS-SATT with FLINT is slightly superior on most problems and performs on the few problems with big numbers well enough to also win overall.

In contrast to rationals and integers, we could not find a satisfactory library for the extension of rationals to δ-rationals. Therefore, we had to implement the δ-rationals and the operations over them ourselves. Our implementation for the δ-rationals and the necessary operations follow the description from Chapter 2.3.2. This means we implemented δ-rationals as a pair of rationals and simply extended the operations over the rationals to δ-rationals.

For the remainder of this thesis, we refer to the hardware integers as int, to the arbitrary precision integers as Integer, to the arbitrary precision rationals as Rational, and to the arbitrary precision δ-rationals as DRational.

Variable Implementation

The variable names we use in linear arithmetic are typically just identifiers. There is, however, other information directly associated with our variables that we want to integrate into their representation. We have, therefore, chosen a more complicated data structure Var for our variables (see Figure 7.15). Naturally, our variables still need an identifier, which is represented by the field "int id". The next field "bool isInteger" marks the type of the variable, i.e., the field is set to true if the variable is an integer variable and to false if it is a rational variable. Var also includes several fields with simplex specific information: The field "bool isBasic" marks whether the variable is basic or non-basic. The field "int idx" represents the index of the variable in the simplex tableau. The index is a row index if the variable is basic and a column index otherwise. The fields "Entry* first, last" represent the first and last non-zero entry in the row/column associated with the variable (the definition of Entry can be found later in this section). Both fields are set to NULL if the row/column does not contain a non-zero entry. The fields "Bound* lower, upper" represent the current lower and upper bound of the variable (the definition of Bound can be found later in this section). This means we have implemented the bound value functions L and U as local fields instead of actual functions. Note that we express the lower (upper) bound value -∞ (∞) by setting the field lower (upper) to NULL. Similarly, we have also implemented the current assignment β and the backup assignment ω as local fields "DRational beta, omega". The field "VarInfo info" stores redundant information (the definition of VarInfo can be found later in this section); This information is redundant because it can also be computed on the fly. However, the computation cost is high enough that it is more efficient to store and incrementally update this information. The final field "TreeSet bounds" is not relevant for SPASS-IQ but for its ex-tension SPASS-SATT. It is an ordered tree set that stores all upper bounds x i ≤ u j constructed for the variable and orders them by their bound values. This ordering establishes a relationship that is especially useful for unate propagation (see Chapter 8).

Bound Implementation

We implement variable bounds x i ≤ b i or x i ≥ b i with the data structure Bound (see Figure 7.15). The first three fields are the most important fields for this chapter: The field "Var* var" defines the variable that is bounded; The field "bool isUpper" defines the bound direction, i.e., whether the bound is an upper or a lower bound; and the field "DRational value" defines the bound value of the bound.

The remaining fields of Bound are not relevant for SPASS-IQ but for its extension SPASS-SATT. The field "int litId" defines the link between the arithmetic representation of a literal as a bound and the boolean representation of a literal as a propositional variable. The field "Bound* negated" is also based on the literal definition and points to the negated version of the bound. This means an upper bound over a rational variable x i ≤ u i points to the lower bound x i ≥ ūi and vice versa. Similarly, an upper bound over an integer variable x i ≤ u i points to the lower bound x i ≥ u i + 1 and vice versa. The final two fields "Bound* smaller" and "Bound* larger" are needed for unate propagation (see Chapter 8). The field "Bound* smaller" points to the largest smaller bound for the same variable and bound direction and the field "Bound* larger" points to the smallest larger bound for the same variable and bound direction. This means the bounds for the same variable and bound direction are linked together in a double-linked list ordered by the bound values. We can efficiently extend this list because of the ordered tree set "TreeSet bounds" stored in the associated variable. This tree set allows us to compute in logarithmic time the next largest/smallest bound in the set and from there we only need a constant number of operations for adding the new bound to the double-linked list.

Tableau Implementation

The simplex tableau Az = y consists of three parts: the coefficient matrix A, the non-basic variables z, and the basic variables y. We implemented the tableau with the data structure Tableau (see Figure 7.16). The Tableau contains arrays "Var[ ] tableau.rows" and "Var[ ] tableau.columns" for our basic and non-basic variables, respectively. The position of a variable var in its respective array corresponds to the variable index var.idx, i.e., tableau.rows[var.idx] = var if var.isBasic = true and tableau.columns[var.idx] = var otherwise. matrix coefficients a ij into integer coefficients. 7 To do so, we compute a common row denominator d i for all matrix coefficients a ij in the row i. Then we multiply all coefficients in the row with the common row denominator and get our integer entry coefficients a ij := a ij • d i , which are saved in the fields "Integer coeff" of the entries. We also save the common row denominators in an array "Integer[ ] rowCoeffs" of the Tableau structure so that we are able to reconstruct our original coefficients. With the common row denominators, we now have 1 + L integers values per row, where L is the number of row entries. With the rational coefficients, we would have 2 • L integers values per row instead (i.e., two per rational coefficient). This means we almost reduced the number of integer operands by half.

As our second step, we sort our row lists and keep them sorted as an invariant. We do so because the row ordering allows us to efficiently identify which columns are shared by two rows, i.e., for which columns both rows have non-zero coefficients. The actual ordering is not really relevant for optimizing row summations it is just important that all rows are sorted by the same well-founded total order. We personally choose for our order that columns[entry.colIdx].id is strictly increasing because this order optimizes our pivoting strategy, which we discuss later in more detail.

As our third step, we forbid any order invariants on the column lists, i.e., we expect that no part of our implementation ever requires that our column lists stay sorted in a specific way. We do so because it allows us to efficiently add a new entry to a given column, i.e., we can add it in constant time to the end of the column.

Based on these three steps, we are now able to define an efficient algorithm AddMultipleOfRowToRow() for row summation (Figure 7.17). 8 It is linear in the number of non-zero entries of both rows and keeps both rows sorted. The algorithm itself is based on the classic merge function of the merge sort algorithm. There are two reasons why the algorithm is linear. First of all, the double-linked nature of our rows and columns allows us to implement the functions RemoveEntry(), CreateEntryBeforeEntry(), and CreateEntryAtEndOfRow() in such a way that they have a constant runtime. (We omit the pseudocode for these functions because their names are descriptive enough.) Secondly, the sorted row lists allow us to efficiently identify which columns are (not) shared by the two rows. has a finite lower/upper bound; isTightAtLower/isTightAtUpper indicates whether the variable has a tight lower/upper bound, i.e., whether the value assigned to the variable matches the lower/upper bound; and isViolating is true iff the variable assignment violates the upper or lower bound of the variable. The second field "int numEntries" counts simply the number of entries between var.first and var.last. The next four fields are only used when the variable is a basic variable. They count the number of non-basic variables that share an entry with this basic variable and fulfill certain properties: "int numLow" considers only non-basic variables with a finite lower bound when the associated entry coefficient is positive and with a finite upper bound when the associated entry coefficient is negative; "int numUpp" considers only non-basic variables with a finite upper bound when the associated entry coefficient is positive and with a finite lower bound when the associated entry coefficient is negative; "int numTightLow" considers only non-basic variables with a tight lower bound when the associated entry coefficient is positive and with a tight upper bound when the associated entry coefficient is negative; and "int numTightUpp" considers only non-basic variables with a tight upper bound when the associated entry coefficient is positive and with a tight lower bound when the associated entry coefficient is negative. The next two fields are also only used when the variable is a basic variable. During a call of the Check() function (see Figure 2.2), "DRational difference" stores the absolute difference between the variable assignment and its violated bound or zero if no bound is violated. Outside of a call to the Check() function, i.e., during a sequence of assertions (see Figure 2.4), "DRational difference" stores the difference between the variable assignment before the sequence and the variable assignment at the moment. During a call of the Check() function (see Figure 2.2), "int direction" indicates what bound is violated, i.e., 0 means no bound is violated, > 0 means the upper bound is violated, and < 0 means the lower bound is violated. Outside of a call to the Check() function, i.e., during a sequence of assertions (see Figure 2.4), "int direction" stores the directions of the newly asserted bounds, i.e., 0 means no new bounds for the variable, 1 and 3 mean a new lower bound has been asserted for the variable, and 2 and 3 mean a new upper bound has been asserted for the variable. The final field "int pos" indicates the position of the variable in the violating variable heap, which we discuss in more detail in Section 7.1.2.

Based on VarInfo, we are now able to compute some properties in constant runtime that were previously only computable in linear runtime. For instance, we can compute in constant time whether a basic variable and its row describe a conflict (see Figure 7.19). We previously defined a conflict by the statements in lines 7 and 13 of the Check() function (see Figure 2.2). These lines correspond to the checks in function IsConflict() (see Figure 7.19).

The main structure of the branch-and-bound algorithm is a so-called branching tree. We implement branching trees in SPASS-IQ with the help of the data structure BNode (Figure 7.20). Each BNode represents a node in a branching tree. The field "BNode* parent" points to the parent node of the given node and it points only to NULL if the given node is a root node. The field "int depth" stores the depth of the given node in its branching tree. The field "bool branched" is true when the node has been branched, i.e., our current node is the parent node for some other nodes. The field "BNodeType type" is the type of the given node and states whether the node is the root node ("rootNode") or whether the node was generated because of a lower or upper branch ("lowerBranch" or "upperBranch"). All nodes that point to the same parent node have different types, i.e., a node has at most one lower and one upper branch as its children. If the branch-and-bound method creates branches for the given node, then the field "Var* bvar" points to the selected branching variable and the field "DRational bval" stores the branching value, i.e., the assignment of the branching variable at the moment of branching.

Based on the BNode structure, we are now able to represent the whole branching tree BTree (Figure 7.20 through one root node (BNode* root) and one dynamically growing array that stores all active nodes (BNode* activeNodes).

Combining Pruning Conflicts

We mentioned before that a theory solver should return a conflict, i.e., an unsatisfiable subset of asserted constraints, if the problem is unsatisfiable. We construct such a conflict in our branch-and-bound implementation by combining the rational conflicts the simplex solver returns for each pruned node. This allows us to recursively generate one conflict for the whole branching tree. The problems associated with our nodes do, however, contain branching bounds and propagated bounds in addition to the original constraints from our input problem. We do not want these bounds in our combined conflict because we would otherwise need to add new literals on the fly, which is also the reason why we avoided splitting-on-demand.

In order to avoid all but the original constraints, we eliminate propagated bounds in all explanations by replacing them with their propagation explanation, i.e., a set of constraints that imply the propagation of the bound (see Chapter 2.7.2). This also means that we replace previously propagated bounds in the propagation explanation of a newly propagated bound. Thereby, all conflict/pruning and propagation explanations contain only branching bounds and original constraints.

Next, we simultaneously combine conflict/pruning explanations and remove the branching bounds from our explanations. For this purpose, let us look at the abstract recursive case of the procedure. As our input we get (i) a node C (ii) that has two child nodes C l := C ∪ {x i ≥ c i } and C u := C ∪ {x i ≤ c i } (iii) such that C l and C u are the (combined) conflict explanations for C l and C u , respectively. As the invariant of our procedure, we assume that C l ⊆ C l , i.e., C l contains only constraints that also appear in C l , and C u ⊆ C u , i.e., C u contains only constraints that also appear in C u . Based on this invariant, we have to differentiate three cases: Case 1: C l does not contain the branching bound x i ≥ c i , which means that C l is also a conflict for C because C l ⊂ C. Therefore, C := C l is the combined conflict for C and we can stop. Case 2: C u does not contain the branching bound x i ≤ c i , which means that C u is also a conflict for C because C u ⊂ C. Therefore, C := C u is the combined conflict for C and we can stop. Case 3: C l contains the branching bound x i ≥ c i and C u contains the branching bound x i ≤ c i . Since x i ≥ c i is mixed equivalent to ¬(x i ≤ c i ), we can apply boolean resolution to combine C l and C u . The result, C := (C l ∪ C u ) \ {x i ≥ c i , x i ≤ c i }, is then the combined conflict for C and still mixed unsatisfiable. In all three cases, the invariant is maintained, i.e., the resulting combine conflict C contains only constraints that also appear in C. This also means that the combined explanation C 0 for the root node C 0 contains only original constraints.

Branch-And-Bound Selection Strategies

The branch-and-bound algorithm as defined in Chapter 2.7.3 is non-deterministic, e.g., with regard to the active node selection and the selection of a branching variable. Here we present the strategies that SPASS-IQ uses to resolve the non-deterministic selection. Moreover, we explain whether the choice of active node and branching variable has a measurable impact on the runtime performance of branch-and-bound.

Active Node Selection

At the beginning of every iteration of the branch-and-bound algorithm, an active node (corresponding to a set of asserted bounds C) has to be selected. Naturally, there are many different selection strategies [START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF], but the most obvious one is depth-first.

We distinguish three cases for the depth-first selection strategy:

• if the root node C 0 is active, then we select it next;

• if branch-and-bound has created two child nodes C l and C u for the last selected node C, then we select either of them next; • if the last selected node is pruned, then we select one of the active nodes with the largest number of ancestor nodes next.

The depth-first selection strategy still has some non-determinism because it does not say which child is selected in the second case 9 . In our own implementation of branch-and-bound, we resolve this non-determinism with the following case distinction on the sets of asserted bounds corresponding to the nodes C l := C ∪ {x j ≥ β(x j ) } and C u := C ∪ {x j ≤ β(x j ) }:

• if x j is fixed in C l , i.e., there exists v j ∈ Q δ such that {x j = v j } ⊆ C l , but not fixed in C u , i.e., {x j = v j } ⊆ C u for all v j ∈ Q δ , then we select C l next; • if x j is fixed in C u but not fixed in C l , then we select C u next;

• if both or none of the branches fix x j , then we choose C l if the input constraint set C 0 contains more constraints of the form a ij x j + p ij ≤ 0 with a ij > 0 than a ij < 0, and otherwise C u . 10The number of constraints of the form a ij x j + p ij ≤ 0 with a ij > 0 is a good indicator for the potential number of bound propagations between the nodes C and C u . The number of constraints of the form a ij x j + p ij ≤ 0 with a ij < 0 is a good indicator for the potential number of bound propagations between the nodes C and C l . This means we try to select the branch with the smallest number of bound propagations. This, in turn, is a good indicator for the rational satisfiability of the selected node.

As mentioned before, there exist many other active node selection strategies and we compared many of them in practice. We were, however, unable to find a strategy that made a significant difference in performance. The only exception is the node we select directly after a branching. For this specific case, our proposed rule seems to give us a relevant advantage. We, therefore, conclude that it is more relevant to find a good successor after a successful branching than after a pruning.

Branching Variable Selection

Branch-and-bound also has to select a branching variable x j for every selected node with a rational but not a mixed solution. Again there are many different selection strategies [START_REF] Jünger | 50 Years of Integer Programming 1958-2008 -From the Early Years to the State-of-the-Art[END_REF][START_REF] Schrijver | Theory of linear and integer programming[END_REF].

In our own implementation of branch-and-bound, we select the most fractional integer variable. A variable x j is more fractional than variable x k under assignment β(x) := s, if: |s js j | > |s ks j | . Any remaining non-determinism is resolved with an order over the variables.

We selected this strategy after comparing several other branching variable selection strategies in practice. This includes selecting the minimal fractional integer variable, several fixed variable orders, and other more complex strategies. We were, however, unable to find a strategy that performed better. (Although the fixed variable orders performed worse.)

Extensions to Branch-And-Bound

We have already presented the most popular branch-and-bound extensions used by the SMT community in Chapter 2.7.4. Moreover, we have presented several of our own branch-and-bound extensions in the previous chapters. In this subsection, we will give again a short summary on each of the branchand-bound extensions implemented in SPASS-IQ. The extensions used by SPASS-IQ are: a simple rounding test (Chapter 2.7.4), a unit cube test (Chapter 4), a bound propagation procedure (Chapters 2.7.2 and 8.1.2), and our two transformations that reduce any problem into a bounded problem (Chapter 6). The exact way in which they are integrated in SPASS-IQ and in which they are combined with branch-and-bound will be explained in the next subsection.

Simple Rounding

Simple Rounding (Chapter 2.7.4) is an easy way to turn any rational solution β(x) = s computed during branch-and-bound, into a mixed assignment s ∈ Q n 1 δ × Z n 2 that fulfills the types of all variables. We simply round all s j with j > n 1 to an integer value s j = s j and keep all other values as before. Beware that this new assignment might very well violate the node or input constraints. We, therefore, have to evaluate whether s is a mixed solution for our original problem C 0 . If the heuristic solution s satisfies the original problem, then we stop our branch-and-bound search and return the solution instead. Otherwise, branch-and-bound ignores s and continues the branch-and-bound search with s. This technique can also be found in other SMT theory solvers, e.g., in Yices [START_REF] Dutertre | Yices 2.2[END_REF].

Unit Cube Test

The unit cube test (Chapter 4) determines in polynomial time whether a polyhedron, i.e., the geometric representation of a system of inequalities, contains a hypercube parallel to the coordinate axes with edge length one. This information is useful because such a hypercube guarantees a mixed/integer solution for the system of inequalities.

The unit cube test is only a sound and not a complete decision procedure. However, there is at least one class of inequality systems, viz., absolutely unbounded inequality systems (see also Chapter 2.8), that are trivial for the unit cube test and much harder for many complete decision procedures. These problems are also the reason why we extend our own branch-andbound implementation with the unit cube test.

The unit cube test itself replaces only the bounds in our original system of inequalities. The resulting problem over the theory of linear rational arithmetic is then solved internally with our simplex implementation. Note also that our implementation of the simplex algorithm is designed to efficiently exchange bounds (see Chapter 2.7). After applying the test, we can use this design to easily recover the original system by reverting to the original bounds. In doing so, the unit cube test conserves the incremental connection to the different original systems. It is, therefore, an incrementally efficient technique.

Bound Propagation

Bound propagation (Chapter 2.7.2) is a technique for linear arithmetic but in itself not a decision procedure. 11 It is mainly used to extend other decision procedure so they perform better in practice. Bound refinement takes as input an inequality a ij x j + p ij ≤ 0 and several variable bounds from our current set of constraints C. Then it tries to propagate an entailed variable bound for (at least) one of the variables x j with non-zero coefficient a ij . In the case that bound refinement succeeds, it produces a variable bound x j ≤ u j (x j ≥ l j ) that is not subsumed by an existing bound in C. 12 The produced bound x j ≤ u j (x j ≥ l j ) can then be explicitly added to C. The computation of the entailed bounds is described in detail in Chapters 2.7.2.

For linear mixed/integer arithmetic, we combine bound propagation with constraint tightening. This means we round any bound x j ≤ u j (x j ≥ l j ) for an integer variable x j to its closest integer value x j ≤ u j (x j ≥ l j ). In many cases, bound refinement combined with constraint tightening is enough to cutaway some of the rational solutions while keeping all of the mixed/integer solutions. These cuts are not very strong but they have the advantage that they always subsume another inequality/bound. Thus, bound propagation does not increase the number of constraints and, therefore, not the time for computing a rational solution. In SPASS-IQ, we always perform a bounded number of bound propagations before calculating a rational solution for a new branch.

Branch-and-bound alone is an incomplete decision procedure and we already explained in Chapter 2.8 that the unboundedness of a problem indicates whether branch-and-bound terminates. However, we also presented two transformations in Chapter 6 (the Mixed-Echelon-Hermite transformation and the Double-Bounded reduction) that reduce any unbounded problem into a mixed equisatisfiable problem that is bounded. The transformed problem can then be solved with our branch-and-bound implementation because it is complete for bounded problems.

Although the transformed problem is only mixed equisatisfiable, there still exist ways (Corollary 6.2.9 & Lemma 6.3.5) to efficiently convert certificates of (un)satisfiability between the transformed and the original system. Our method is efficient, compared to other extensions that complete branch-and-bound, because it is fully guided by the structure of the problem. Moreover, it can be implemented in an incrementally efficient way (see Section 6.4).

Finding Unbounded Directions Efficiently

The Mixed-Echelon-Hermite transformation and the Double-Bounded reduction are useful to prevent branch-and-bound from diverging. However, they might reduce the efficiency of branch-and-bound when applied to already bounded problems. We, therefore, use our bounded basis techniques from Chapter 5.5 to efficiently determine whether an input problem is (un)bounded and which directions are (un)bounded.

An Efficient Combination of Techniques

We combine in SPASS-IQ several decision procedures in order to handle problems over the theory of linear mixed/integer arithmetic. This combination of decision procedures is divided into two parts: the preprocessing procedures, i.e., the procedures we apply once before we actually apply branch-andbound, and the inprocessing procedures, i.e., the procedures we apply during every iteration of branch-and-bound.

The preprocessing procedures (Figure 7.21) have two purposes: they either prevent us from unnecessarily constructing a branching tree because there exists an easy to find mixed solution; or they transform the input problem because it is otherwise too hard for branch-and-bound. To this end, we first search for a rational solution with the simplex algorithm. If we already fail to find a rational solution, then there cannot exist a mixed solution. Therefore, we can stop and return unsatisfiable with the conflict from the In case one, the propagation results in two asserted bounds for a variable x i that are contradicting (i.e., U(x i ) < L(x i )). This means that the subproblem C has no mixed solution and we can prune the selected node. Next we try to select one of the still active nodes based on the selection strategy presented in Subsection 7.2.2. If there are no more active nodes, then we have proven that the input problem has no mixed solution and we can return unsatisfiable together with a conflict constructed as described in Subsection 7.2.1. Otherwise we select and remove one of the active nodes from the set of active nodes, and continue with a new iteration of branch-and-bound.

In case two, the propagation did not result in two conflicting bounds and we continue our execution by applying the simplex algorithm to the subproblem C extended by the propagated bounds, i.e., C ∪ C . Depending on the result of the simplex algorithm, we have to distinguish again between two cases. We prune the selected node in the case that the simplex algorithm returns unsatisfiable. As in the previous pruning case, we now try to select one of the still active nodes based on the selection strategy presented in Subsection 7.2.2. If there are no more active nodes, then we have proven that the input problem has no mixed solution and we can return unsatisfiable together with a conflict constructed as described in Subsection 7.2.1. Otherwise we select and remove one of the active nodes from the set of active nodes, and continue with a new iteration of branch-and-bound.

In the case that the simplex algorithm returns satisfiable, we get at least an assignment β that is a rational solution for C ∪ C and, therefore, also for C. We use this rational solution β for a simple rounding test (Chapter 2.7.4) and determine whether the rational solution rounded to a mixed assignment β is also a mixed solution. 14 If it is a mixed solution, then we can stop branch-and-bound and return satisfiable because the mixed solution for the subproblem is also a mixed solution for the original problem.

If the mixed assignment is not a solution, then we select a branching variable x i based on the rational solution β (see Subsection 7.2.2). Next we branch the node C into two new nodes that represent the subproblems

C l := C ∪ C ∪ x i ≥ β(x i ) and C u := C ∪ C ∪ x i ≤ β(x i ) , respectively.
Our algorithm then selects one of the nodes C l and C u as the next selected node and adds the other one to the list of active nodes (see Subsection 7.2.2 for the selection strategy). We continue with the newly selected node in a new iteration of branch-and-bound.

Multiple Tableau Instances

The basis of our implementation is the simplex algorithm and the basic representation of our constraints in the simplex implementation was through the Tableau structure. The simplex algorithm does, however, change the coefficient matrix of the Tableau and, thereby, the explicit constraints it represents. These changes, also called pivots, are equivalence preserving so our original constraints are still implicitly represented. We need, however, the original versions of the constraints for some of our decision procedures, i.e., the simple rounding test (Chapter 2.7.4), the unit cube test (Chapter 4), and the Mixed-Echelon-Hermite transformation (Chapter 6). To be more precise, we need the original versions in order to (i) test the rounded assignments from our simple rounding test efficiently; (ii) compute the changed bound values for the unit cube test based on the original coefficient matrix; (iii) transform the original coefficient matrix into its Mixed-Echelon-Hermite normal form.

For this purpose, we have at all times two instances of the Tableau structure: the static instance and the dynamic instance. The static instance explicitly represents our original constraints and is never pivoted nor do we add branching or propagating bounds to it. The dynamic instance implicitly represents our original constraints and we use it for our actual simplex computations. 15We need three more instances of the Tableau structure when we encounter a partially unbounded problem. 16 The first and second instances are again static and dynamic instances, but this time they represent our original constraints after the Mixed-Echelon-Hermite transformation. The static instance explicitly represents our original transformed constraints and is never pivoted nor do we add branching or propagating bounds to it. It is used for the efficient simple rounding test on the transformed constraints and it is needed for the iterative extension of the Mixed-Echelon-Hermite transformation. 17 The dynamic instance implicitly represents our transformed constraints and we use it for our actual simplex computations on the transformed system. The third instance represents the column transformation matrix for our Mixed-Echelon-Hermite transformation. It is needed 10 -2 10 -1 10 0 for the efficient solution conversion between the original and the transformed system and for the iterative extension of the Mixed-Echelon-Hermite transformation. (See Chapter 6.4.3 for more details on why we need these instances for the iterative extension of the Mixed-Echelon-Hermite transformation.)

Branch-And-Bound Experiments

In order to measure the impact of our various branch-and-bound extensions, we performed a series of benchmark experiments with SPASS-SATT, the CDCL(LA) extension of SPASS-IQ. Our first experiment (see Figure 7.23) in this series of experiments examines the impact that the combination of our extensions have on SPASS-SATT's performance on the QF LIA benchmarks. To this end, we compare the default version of SPASS-SATT with all four extensions (horizontal axis) and the version of SPASS-SATT without any of the extensions (vertical axis). Clearly, our extensions improve the number of solved instances by a significant amount. This amount becomes even more significant if we look at Figure 7.24 that illustrates how rarely we actually need branch-and-bound at all (641 out of 6947 instances 10 -2 10 -1 10 0 With BnB Runtime (seconds) Without BnB Runtime (seconds) QF LIA with(out) Branch-And-Bound (BnB) SPASS-SATT with branch-and-bound solves 6806 instances SPASS-SATT without branch-and-bound solves 6165 instances Figure 7.24: With(out) branch-and-bound (BnB) on the QF LRA SMT-LIB benchmarks or 1912 out of 6947 instances if we also turn off the unit cube test). 18 In fact, SPASS-SATT needs branch-and-bound only for the benchmark families 20180326-Bromberger (for 200 out of 806 instances or 618 out of 806 instances if we also turn off the unit cube test), arctic-matrix (for 3 out of 100 instances), calypto (for 21 out of 37 instances), CAV-2009 (for 9 out of 591 instances or 465 out of 591 if we also turn off the unit cube test), CIRC (for 11 out of 51 instances), (can only solve 36 of them) convert (for 282 out of 319 instances), cut lemmas (for 69 out of 93 instances), dillig (for 4 out of 233 instances or 216 out of 233 if we also turn off the unit cube test), miplib2003 (for 2 out of 16 instances), prime-cone (for 18 out of 27 instances or 26 out of 27 if we also turn off the unit cube test), slacks (for 4 out of 233 instances or 204 out of 233 if we also turn off the unit cube test), tightrhombus (for 12 out of 12 instances), and tropical-matrix (for 6 out of 107 instances).

The remaining figures in this section illustrate the impact of each of our four branch-and-bound extensions separately. In it, we compare the default version of SPASS-SATT with simple rounding (horizontal axis; command-line option -LASR 1) and the version of SPASS-SATT without simple rounding (vertical axis; command-line option -LASR 0). The plot shows that simple rounding has no impact on the QF LIA benchmarks with the exception of one single problem, which belongs to the miplib2010 benchmark family.

We mentioned before that we put a threshold on the number of bound propagations. To be more precise, we limit at every branching node the number of bounds we propagate per variable. In Figure 7.26, we examine the impact that different bound propagation thresholds have on SPASS-SATT's performance on the QF LIA benchmarks. In the legends of the plots, we list the different bound propagation settings, i.e., the thresholds on the maximum number of bounds we propagate per branching node and variable (command-line option -LABP <threshold>). Moreover, the two numbers in brackets behind each threshold are the numbers of satisfiable and unsatisfiable instances that we can solve with the respective threshold. Naturally, a threshold of 0 means that SPASS-SATT performs no bound propagations and a threshold of ∞ means that SPASS-SATT performs arbitrarily many bound propagations.

The figures show that activating bound propagation (i.e., selecting a nonzero threshold value) is beneficial, although it has no major impact because SPASS-SATT solves only a few more problems. Moreover, the results for the different positive threshold values are too close and can be attributed to performance fluctuations of our cluster. Therefore, we assume that the actual threshold on the number of bound propagations is irrelevant as long as it is non-zero. This observation changes if we also turn of our bounding transformations. If we turn them off, then SPASS-SATT might waste time on unnecessary propagations if the threshold is set too large and might even diverge on some instances if the threshold is set to ∞.

Unit Cube Test

We discuss unit cube tests in more detail in Chapter 4. In that Chapter, we present several benchmark experiments over the unit cube test that focus on absolutely unbounded problems. In Figure 7.27, we examine the impact that the unit cube test has on SPASS-SATT's performance on all QF LIA benchmarks. To do so, we compare the default version of SPASS-SATT employing the unit cube test (horizontal axis; command-line option -C 1) and the version of SPASS-SATT without the unit cube test (vertical axis; command-line 10 -2 10 -1 10 0 What the plot does not show is that there are actually over 1483 instances of absolutely unbounded problems in the QF LIA benchmarks 19 and that SPASS-SATT without unit cube tests is still efficient enough to solve the majority of them on its own. However, this is only possible because the majority of absolutely unbounded problems in the QF LIA benchmarks consist of a reasonably small number of variables and constraints. Therefore, general improvements to SPASS-SATT's simplex and branch-and-bound implementation (e.g. bound propagation and simple rounding) are enough to handle these problems. However, these general improvements do not scale 19 These instances are 229 of the 233 dillig benchmarks designed by Dillig et al. [START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF], 503 of the 591 CAV-2009 benchmarks also by Dillig et al. [START_REF] Dillig | Cuts from proofs: A complete and practical technique for solving linear inequalities over integers[END_REF], 229 of the 233 slacks benchmarks which are the dillig benchmarks extended with slack variables [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], 503 of the 806 20180326-Bromberger benchmarks which include among other benchmarks the CAV-2009 benchmarks extended with slack variables [START_REF] Bromberger | A reduction from unbounded linear mixed arithmetic problems into bounded problems[END_REF], and 19 of the 37 prime-cone benchmarks, that is, "a group of crafted benchmarks encoding a tight n-dimensional cone around the point whose coordinates are the first n prime numbers" [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF].

on larger absolutely unbounded problems, which can also be observed in the plot. In comparison, the unit cube test scales on absolutely unbounded problems and stays efficient for much larger absolutely unbounded problems than those found in the QF LIA benchmarks.

Reduction into Bounded Problems

In Chapter 4, we have presented two transformations (the Mixed-Echelon-Hermite transformation and the Double-Bounded reduction) that reduce any unbounded problem into a mixed equisatisfiable problem that is bounded. For this reason, we also call the combination of these two transformations a bounding transformation. We need a bounding transformation because branch-and-bound alone is only guaranteed to terminate on bounded problems. In Figure 7.27, we examine the impact that the bounding transformation has on SPASS-SATT's performance on all QF LIA benchmarks. To do so, we compare the default version of SPASS-SATT employing the bounding transformation (horizontal axis; command-line option -B 1) and the version of SPASS-SATT without the bounding transformation (vertical axis; command-line option -B 0). The plot shows that SPASS-SATT can solve 169 additional benchmark instances from the QF LIA benchmarks if it employs the bounding transformation. 20 Moreover, the bounding transformation causes only a minor, almost immeasurable overhead on problem instances where it is not successfully applicable. and simplification of if-then-else expressions. This includes simplifications for constant if-then-else expressions, compression of if-then-else expressions, lifting shared terms in nested if-then-else expressions, bounding constant if-then-else expressions, as well as the reconstruction of naively eliminated if-then-else expressions.

With the shared term representation, we can also perform some other preprocessing techniques more efficiently, e.g., the elimination of certain variables through substitution (see Subsection 8.2.4) and the transformation of certain inequalities into clauses (see Subsection 8.2.4). Moreover, our shared term representation brings all arithmetic inequalities into a canonical form.

The last preprocessing technique that we mention is the small-clausenormal-form transformation (see Subsection 8.2.4). It is also the only unique feature of SPASS-SATT that we present in this chapter. 1 All other techniques presented in this chapter have already been available in other SMT solvers such as CVC4 [START_REF] Barrett | CVC4. In CAV[END_REF], MathSAT [START_REF] Cimatti | The MathSAT5 SMT solver[END_REF], Yices [START_REF] Dutertre | Yices 2.2[END_REF], and Z3 [START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF], but not all in one tool. Although these techniques are contained in existing SMT solvers, not all have been described in the respective literature.

Experimental Setup

For this chapter, we have performed several experiments in order to estimate the impact of SPASS-SATT's various features. The benchmarks for our experiments are the 1649 SMT-LIB benchmarks for quantifier free linear rational arithmetic (QF LRA) and the 6947 SMT-LIB benchmarks for quantifier free linear integer arithmetic (QF LIA) from the SMT-LIB (satisfiability modulo theories library) [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF]. All experiments compare different configurations of SPASS-SATT with each other. The experiments are preformed on a Debian Linux cluster from which we allotted one core of an Intel Xeon E5620 (2.4 GHz) processor, 8 GB RAM, and 40 minutes to each combination of problem and SPASS-SATT configuration.

CDCL(LA) Implementation

Our implementation of CDCL(LA) combines SPASS-IQ, our theory solver for ground and conjunctive linear arithmetic formulae, with a CDCL-based (conflict-driven-clause-learning-based) SAT solver from the SPASS Workbench [START_REF]The SPASS Workbench[END_REF]. 2 The result is a decision procedure for ground linear arithmetic formulae in clause normal form. In this section, we will not describe our theory solver and SAT solver as separate entities. 3 Instead, we explain how our theory solver and SAT solver interact in our CDCL(LA) implementation.

Interaction Between SAT and Theory Solver

The CDCL(LA) implementation in SPASS-SATT is actually divided into four phases (Figure 8.1). Three of those phases-propagation, decision, and conflict-are handled mostly by the CDCL-based SAT solver with some minor interactions with the theory solver. The fourth phase-theory verification-is handled completely by the theory solver.

CDCL(LA) loops between these four phases until it determines whether the problem is satisfiable or unsatisfiable. Each iteration of the loop starts with the (unit) propagation phase (Figure 8.2). At the start of CDCL(LA), unit propagation gets the original clauses and an empty model as its input. In later iterations, it receives as its input an equisatisfiable set of clauses and a partial model that is satisfiable on the propositional level. The propagation phase itself is also an iterative process. In every iteration of the propagation phase, the SAT solver first checks whether there is a conflict clause, i.e., whether one of the clauses is unsatisfiable under the current model. 4 If there is a conflict clause, then CDCL(LA) exits the propagation phase and handles the conflict clause in the conflict phase. Otherwise, the SAT solver checks whether a unit propagation is possible, i.e., whether there exists a clause L∨C such that L is still undecided (i.e., not assigned to a truth value) and all literals in C are assigned to false. 5 If there is no such clause, then CDCL(LA) exits the propagation phase and sends the current model to the theory solver for theory verification. Otherwise, the SAT solver propagates L, i.e., the SAT solver extends the model by setting L to true. Moreover, CDCL(LA) notifies the theory solver, so it can assert the corresponding bound to the literal L. The theory assertion is either successful and we continue at the start of the propagation phase or the theory assertion returns a conflict clause. In the latter case, CDCL(LA) exits again the propagation phase and handles the conflict clause in the conflict phase.

The theory verification phase takes the current (partial or full) model as its input. At the start of this phase, the theory solver checks whether its conflict backlog contains any previously found but not yet analyzed conflict explanations (sets of literals). If the backlog contains explanations, then the theory solver searches for an explanation that is still a conflict, i.e., an expla- The decision phase takes the current clause set and the current (partial or full) model as its input. At the start of this phase, the SAT solver checks whether the current model satisfies all clauses. 8 If the current model satisfies all clauses, then CDCL(LA) has determined that the original clause set was satisfiable and it, therefore, returns satisfiable. Otherwise, at least one propositional variable appearing in the current clause set is not yet assigned a truth-value and the SAT solver selects one of them based on a heuristic. If the selected propositional variable corresponds to one of the original propositional variables, then the SAT solver uses a phase saving scheme to decide, whether the propositional variable should be assigned to true or false. If the selected propositional variable corresponds to a bound, then the SAT solver asks the theory solver, whether the propositional variable should be assigned to true or false. The theory solver then recommends a truth value for the propositional variable that corresponds to (i) a bound that cannot cause a conflict if asserted by the theory solver (this is always possible) and (ii) that is (if possible) satisfied by the current assignment for the arithmetic variables. We also call this technique a decision recommendation. Next the SAT solver and theory solver create a backtrack point and then they actually assign/assert the selected propositional variable to the selected truth value. CDCL(LA) then exits the theory verification phase and continues in the propagation phase with the new model.

The conflict phase takes a conflict clause C as input as well as the current set of clauses and the current model. The SAT solver starts the conflict phase with a conflict analysis based on UIP (unique implication point) resolution [START_REF] João | GRASP -a new search algorithm for satisfiability[END_REF][START_REF] Zhang | Efficient conflict driven learning in a Boolean satisfiability solver[END_REF]. This conflict analysis tries to find a previous decision L (i.e., decided literal) that was responsible for the detected conflict. If there exists no such decision, then the original set of clauses is unsatisfiable and CDCL(LA) returns unsatisfiable. Otherwise, CDCL(LA) backtracks to the backtrack point before the decision, i.e., the SAT solver reduces the current model to the prefix without the decided literal and the theory solver reverts the asserted bound values and recomputes a rational solution. The conflict analysis also transforms the conflict clause into a clause L∨C implied by the original clause set such that all literals in C are false under the backtracked model. The SAT solver learns this clause, i.e., adds it to the current set of clauses, after which CDCL(LA) exits the conflict phase and starts again with the propagation phase. The learned clause L ∨ C prevents that L is decided again because it must be used to propagate L.

Key Changes to the General CDCL(T) Framework

There are four key points that we have changed in SPASS-SATT compared to the more general frameworks for CDCL(T) (see Chapter 2.6). 9 First of all, we rely on "weakened early pruning" [START_REF] Sebastiani | Lazy satisability modulo theories[END_REF], i.e., we only use a weaker but faster check to determine theory satisfiability for partial (propositionally abstracted) models. We do so because checking for an integer solution is too expensive and not incrementally efficient enough to be done more than once per complete (propositionally abstracted) model. As a compromise, we at least check whether the partial model has a rational solution, before we add a(nother) decision literal to the model. (Weakened early pruning is used by most SMT solvers [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].)

As our second key change, we let the theory solver select the phase of the next decision literal L, i.e., whether the SAT solver will add the positive or the negated version of L to the model. We call this technique a decision recommendation. (The SMT solver Yices also uses decision recommendations [START_REF] Dutertre | Yices 2.2[END_REF].)

As our third key change, we allow our theory solver to return more than one conflict per theory check. (As mentioned in Chapter 7.1.2, CVC4's theory solver can also return multiple conflicts [START_REF] Barrett | CVC4. In CAV[END_REF].)

Finally, we use theory reasoning to find and learn new clauses implied by the input formula. The reasoning techniques we use for this purpose are unate propagations and bound refinements as proposed in [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]. (Unate propagations and bound refinements are used by most SMT solvers [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].) 9 All of these changes appeared first in other SMT solvers.

CDCL(LA) Experiments

In this section, we evaluate the impact of decision recommendations, unate propagations, and bound refinements. In order to measure their impact, we performed a series of benchmark experiments with SPASS-SATT.

Decision Recommendation

In a previous part of this section, we explained decision recommendations, i.e., an interaction technique in our CDCL(LA) implementation that lets our theory solver decide whether the SAT solver assigns a decision literal to true or false. Figures 8.6 and 8.7 examine the impact that decision recommendations have on SPASS-SATT's performance on the QF LRA and QF LIA benchmarks. To this end, we compare the default version of SPASS-SATT with decision recommendations (horizontal axis; command-line option -p 0) and the version of SPASS-SATT without decision recommendations (vertical axis; command-line option -p 1). SPASS-SATT with decision recommendations is on average better in both benchmark divisions.

On the QF LRA benchmarks, SPASS-SATT with decision recommendations can solve 11 more problems than SPASS-SATT without decision recommendations-1 more problem form the 2017-Heizmann-UltimateIn-variantSynthesis benchmark family, 10 more problems from the Lasso-10 -2 10 -1 10 0 Ranker benchmark family, and 2 more problems from the miplib benchmark family, although also 2 less problems from the tropical-matrix benchmark family. Moreover, SPASS-SATT with decision recommendations is more than twice as fast as SPASS-SATT without decision recommendations on 160 problems from the QF LRA benchmarks. For comparison, SPASS-SATT without decision recommendations is more than twice as fast as SPASS-SATT with decision recommendations on only 50 problems from the QF LRA benchmarks.

On the QF LIA benchmarks, SPASS-SATT with decision recommendations can solve 129 more problems than SPASS-SATT without decision recommendations-1 more problem from the bofill-scheduling benchmark family, 116 more problems from the convert benchmark family, 1 more problem from the miplib2003 benchmark family, 11 more problems form the tropical-matrix benchmark family, 2 more problems from the nec smt benchmark family, but 1 less from both the 201803026-Bromberger benchmark family and the arctic-matrix benchmark family. Moreover, SPASS-SATT with decision recommendations is more than twice as fast as SPASS-SATT without decision recommendations on 389 problems from the QF LIA benchmarks. For comparison, SPASS-SATT without decision recommendations is more than twice as fast as SPASS-SATT with decision recommendations on only 58 problems from the QF LIA benchmarks. Therefore, we conclude that SPASS-SATT is a very beneficial interaction technique overall, although it is not the optimal strategy for every problem instance.

Unate Propagation

Unate propagation is another technique that we presented in a previous part of this section (for more details see also [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]). Figures 8.8 and 8.9 examine the impact that unate propagation has on SPASS-SATT's performance on the QF LRA and QF LIA benchmarks. To this end, we compare the default version of SPASS-SATT with unate propagations (horizontal axis; command-line option -p 0) and the version of SPASS-SATT without decision recommendations (vertical axis; command-line option -p 1). Although SPASS-SATT frequently and regularly performs unate propagations in both benchmark divisions, we are unable to observe any consistent benefit from this interaction technique. SPASS-SATT with unate propagations cannot solve more benchmark instances than SPASS-SATT without unate propagations (in fact it solves one instance less due to performance fluctuations of our cluster) and any speed-up gained on some of the benchmark instances is countered by a slow-down on other benchmark instances. We still keep unate propagations by default in SPASS-SATT because we assume that there exist some theory combinations where it is beneficial.

Bound Refinements

The final interaction technique that we have mentioned in this section, are bound refinements (see Chapter 2.7.2 for a formal definition). These bound refinements are preformed in addition to the bound propagations of our branch-and-bound solver (see Chapter 7.2.3). However, in contrast to the branch-and-bound bound propagations, the interaction bound refinements propagate only bounds that correspond to already existing literals in the current set of clauses. Moreover, we test more frequently for interaction bound refinements than for branch-and-bound bound propagations. In fact, we test during every theory verification phase for interaction bound refinements, but only during the branch-and-bound search for bound propagations.

The two methods have, however, in common that we put a threshold on the number of bound propagations per refinement/propagation phase. To be more precise, we limit at every branching node the number of bounds we refine/propagate per variable as part of the branch-and-bound bound propagation. Similarly, we limit during every theory verification phase the number of bounds we refine/propagate per variable as part of the interaction bound refinements. In Figure 7 8.9 of this chapter, we now examine the impact that bound refinements have on SPASS-SATT's performance on the QF LRA and QF LIA benchmarks.

In the legends of the plots, we list the different bound refinement settings, i.e., the thresholds on the maximum number of bounds we propagate per theory verification phase and variable (command-line option -LABR <threshold>). Moreover, the two numbers in brackets behind each threshold are the numbers of satisfiable and unsatisfiable instances that we can solve with the respective threshold. Naturally, a threshold of 0 means that SPASS-SATT performs no bound refinements and a threshold of ∞ means that SPASS-SATT performs arbitrarily many bound refinements. The figures show that activating bound refinements (i.e., selecting a nonzero threshold value) is beneficial and allows SPASS-SATT to solve several more problems. Moreover, the results for the different positive threshold values are too close and can be attributed to performance fluctuations of our cluster. Therefore, we assume that the actual threshold on the number of bound refinements is irrelevant as long as it is non-zero.

We also observe, that interaction bound refinements seem to have a much larger impact on the overall performance of SPASS-SATT than branch-andbound bound propagations. For instance, we can solve 24 more problems with interaction bound refinements on the QF LIA benchmarks, but only 8 more problems with branch-and-bound bound propagations.

Preprocessing

The satisfiability modulo theories library (SMT-LIB) [START_REF] Barrett | The Satisfiability Modulo Theories Library (SMT-LIB)[END_REF] offers a large number of benchmarks for ground linear arithmetic formulae and is, therefore, an important resource for testing the efficiency of our decision procedures. All SMT-LIB benchmarks are written in the SMT-LIB language [START_REF] Barrett | The SMT-LIB Standard: Version 2.0[END_REF], which can construct all ground linear arithmetic formulae consisting of the standard boolean operators (and, or, not, implication, equivalence), the standard arithmetic operators (plus, minus, multiplication, division, less than, greater than, less than or equal, greater than or equal, equal), as well as some special operators (let, if-then-else). This is a problem because our CDCL(LA) implementation can only handle ground linear arithmetic formulae in clause normal form. We, therefore, have to preprocess the input problems, given to us in the SMT-LIB language, into an equisatisfiable formula in clause normal form.

The specific preprocessing steps that SPASS-SATT applies before it sends the problem to our CDCL(LA) implementation are outlined in the remainder of this section. But first, let us take a closer look at the SMT-LIB language.

SMT-LIB Language

A linear arithmetic input problem constructed with the SMT-LIB language consists of four sets: a set of propositional variables P , a set of integer variables Z, a set of rational variables Q, and a set of formulae F over those variables (which is semantically interpreted as a conjunction of formulae). The three sets of variables are disjoint and consist of identifiers identifier . The set of formulae consists of formulae formula constructed according to the following subset of the SMT-LIB language:

= digit | nzdigit digit + | (-nzdigit digit + ) rational ::= digit . digit + | nzdigit digit + . digit + | (-digit . digit + ) | (-nzdigit digit + . digit + ) digit ::= 0 | nzdigit nzdigit ::= 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 identifier ::= start char char * letter ::= A | B | C | D | E | F | G | H | I | J | K | L | M | N | O | P | Q | R | S | T | U | V | W | X | Y | Z | a | b | c | d | e | f | g | h | i | j | k | l | m | n | o | p | q | r | s | t | u | v | w | x | y | z | start char ::= letter | ˜| ! | @ | $ | % | ˆ| & | * | | -| + | = | < | > | . | ? | / char ::= start char | digit
The operators in the SMT-LIB language are all prefix operators. The standard boolean operators in the ground linear arithmetic subset are:

• a conjunction operator (and t • a subtraction operator (-t 1 t 2 ), which is semantically interpreted as t 1t 2 ; • an arithmetic negation operator (-t 1 ), which is semantically interpreted as -t 1 ; • a division operator (/ t 1 t 2 ), which is semantically interpreted as t 1 /t 2 with t 2 containing no subterms that are variables. The arithmetic comparison operators in the ground linear arithmetic subset are:

• an equality operator (= t 1 t 2 ), which is semantically interpreted as t 1 = t 2 ; • a greater than operator (> t 1 t 2 ), which is semantically interpreted as t 1 > t 2 ; • a greater than or equal operator (>= t 1 t 2 ), which is semantically interpreted as t 1 ≥ t 2 ; • a less than operator (< t 1 t 2 ), which is semantically interpreted as t 1 < t 2 ; • a less than or equal operator (<= t 1 t 2 ), which is semantically interpreted as t 1 ≤ t 2 . The ground linear arithmetic subset of the SMT-LIB language also contains two special operators. The first special operator is the if-then-else operator (ite t 1 t 2 t 3 ), which is semantically interpreted as the function that returns t 2 if the formula t 1 is true and returns t 3 if the formula t 1 is false. We also assign to the arguments of the if-then-else operator special names so we can better differentiate them. The first argument t 1 is the condition, the second argument t 2 is the consequence, and the third argument t 3 is the alternative. Moreover, we call (ite t 1 t 2 t 3 ) a formula if-then-else expression if its consequence t 2 and alternative t 3 are formulae formula and arithmetic if-then-else expression if its consequence t 2 and alternative t 3 are arithmetic terms la term .

The second special operator is the let operator (let ((x 1 t 1 ) . . . (x n t n ))t). It is actually the most complicated operator that the linear arithmetic subset of the SMT-LIB language contains. It is so complicated because it assigns terms to, potentially new, local variables x 1 , . . . , x n called let variables L. It is semantically interpreted as t{x 1 → t 1 , . . . , x n → t n }, i.e., the term t except that all occurrences of x 1 , . . . , x n in t are substituted by t 1 , . . . , t n , respectively. The only exception happens, when a different let operator in t overwrites the replacement term for one of the variables. For instance, (let ((

x 1 t 1 )(x 2 t 2 ))(let ((x 2 t 3 ))t 4 )) is interpreted as t 4 {x 1 → t 1 , x 2 → t 3 {x 1 → t 1 , x 2 → t 2 }}, i.
e., all occurrences of x 1 and x 2 are replaced in t 4 by t 1 and t 3 , respectively, where t 3 is equivalent to t 3 except that all occurrences of x 1 and x 2 are replaced by t 1 and t 2 . This means that let operators substitute bottom-up, i.e., (let ((x 1 t 1 ) . . . (x n t n ))t) is interpreted as t {x 1 → t 1 , . . . , x n → t n }, where t is equivalent to t except that all let operators in s are already substituted away.

To avoid confusion, let operator can only overwrite other let variables and not operators or propositional, integer, or rational variables. In general, propositional variables, integer variables, rational variables, and let variables cannot be named after operators and all four sets of variables are disjoint.

As mentioned before, SPASS-SATT is given an input problem defined by the above language and it needs to transform it into an equisatisfiable formula in clause normal form. But first of all, this means that SPASS-SATT has to represent these formulae and their subterms internally. In SPASS-SATT we differentiate between four types of terms: constant truth values, variables, numbers, and complex terms. A constant truth value term is either true or false and we can represent them internally as an enumeration structure with two values. Variables are in the input language string identifiers, but we can simplify the identifiers internally to distinct numeral ids. 10 When we are specifying an abstract variable, we simply write p i , i.e., the letter p with and index i, for a propositional variable prop var , x i , i.e., the letter x with and index i, for an arithmetic variable la var , l i , i.e., the letter l with and index i, for a let variable let var , and y i , i.e., the letter y with and index i, for an arbitrary variable. Numbers are also represented by strings in the input language, but internally we use the data types Integer and Rational to represent integer and rational numbers, respectively. When we are specifying an abstract number, we simply write a i , i.e., the letter a with and index i. Complex terms make up all other terms. They consist of two things: an operator (e.g., and, or, +, <=, represented through distinct numeral ids) and an array of arguments, which are pointers to other terms. 11 When we are specifying an abstract complex term, then we simply write it in prefix form (o t 1 . . . t n ), as we did in the input language. Here, o is the operator and t i are the abstract term arguments.

Term Sharing

When we first parse our problem from the input language into internal terms, the resulting set of terms is not yet shared, i.e., two syntactically equivalent subterms are still represented by two different internal terms. We do, however, start to share our internal (sub)terms on the formula level as part of our let operator elimination. The shared term representation has the additional benefit that it saves memory; in some cases even an exponential amount. 12We start our let operator elimination and sharing procedure by pushing negations top-down with the following set of equivalence preserving transformations:

( → (and (not t 1 ) . . .

(not t n )) (=> t 1 t 2 ) → (or (not t 1 ) t 2 ) (not (= t 1 t 2 )) → (= t 1 (not t 2 )) (not (ite t 1 t 2 t 3 )) → (ite t 1 (not t 2 ) (not t 3 )) (ite (not t 1 ) t 2 t 3 ) → (ite t 1 t 3 t 2 ) (ite t 1 false true) → (not t 1 ) (not (let ((l 1 t 1 ) . . . (l n t n ))t)) → (let ((l 1 t 1 ) . . . (l n t n ))(not t))
As a result, the negation operator appears only on top of propositional variables, let variables, and arithmetic comparators (i.e., <=, >=, <, >, =). From now on, we call (not y i ) a negative occurrence of the variable y i and any occurrence of y i that is not below a negation a positive occurrence.

Next we traverse the sub-terms in all formulae f from our set of formulae F bottom-up. For each let operator t := (let ((l 1 t 1 ) . . . (l n t n ))t) that we find in f , (i) we replace all t by t in f , (ii) we create terms (not t i ) for all t i if t i is a formula , (iii) we push the negations top-down in the new terms (not t i ) so we get equivalent terms t i , (iv) we transform the unshared terms t i and t i into shared terms s i and s i (will be explained in more detail later), and (v) we link l i with its positive shared replacement s i and its negative shared replacement s i in a map M . Finally, we also transform the unshared formulae f from our set of formulae F into shared formulae.

Transforming an unshared term t into a shared term works as follows: First we need three maps. The first map M , was discussed previously, and contains the shared replacement terms s i , s i for all let variables l i occurring in t. M already contains these replacements because we first traversed and shared the let operations bottom up. The second map V maps all truth values, as well as all propositional and arithmetic variable ids to shared term versions. This map and the contained shared terms are constructed after we push-down the negations. The third map S maps all already shared terms s to their superterm set, i.e., a set that contains all shared terms s that have s as an argument. This map will be filled and updated as part of the sharing transformation itself. Now with these three maps in hand, we traverse the subterms t of t bottom-up and replace them by shared versions. If t := l i is a positive occurrence of a let variable term l i , then we get its positive shared replacement s i through the hash map M . If t := (not l i ) is a negative occurrence of a let variable term l i , then we get its negative shared replacement s i through the hash map M . If t := y i is an arithmetic or propositional variable, then we lookup the shared replacement term from V . If t is a truth value, then we also lookup the shared replacement term from V . If t := (o s 1 . . . s n ) is a complex term, then we look through the superterm set of s 1 and check whether there already exists a complex but shared version of the term (o s 1 . . . s n ), i.e., a shared term s with the same operator o and argument pointers s 1 . . . s n , which can be used as the shared replacement term t . If there exists no such shared term s , then t itself becomes our shared replacement term, we add it to the superterm sets of s 1 . . . s n , and we map t in S to a new and empty set.

We continue our traversal in this way, until the original term t is also a shared version of itself. Our transformed formula needs still the same order of size as the original formula, although it has no more let operations, because we combined the let-elimination with the sharing process.

An Alternative Let-Elimination

The above described standard let-elimination causes sometimes an exponential blow-up in the formula size. For the moment, our implementation is not hindered by this blow-up because there are still only linearly many different subterms, which can be represented in a linear amount of memory by our shared term representation. The blow-up will, however, impact the implementation when it has to perform the CNF transformation and the CDCL(LA) algorithm.

We could actually prevent the blow-up of the formula size with an alternative let-elimination. For this alternative let-elimination, we traverse the sub-terms in all formulae f from our set of formulae F bottom-up. For each let operator t := (let ((l 1 t 1 ) . . . (l n t n ))t) that we find in f , (i) we turn the let variables l i corresponding to formula terms t i into propositional variables, (ii) we turn the let variables l i corresponding to arithmetic terms t i into arithmetic variables, (iii) we replace all t by t in f , (iv) we extend our formula set to F := F ∪ {(= l 1 t 1 ), . . . , (= l 1 t 1 )}. Finally, we transform the formulae f from our set of formulae F into shared formulae.

The above described alternative let-elimination would prevent the potential exponential blow-up in the formula size. It would, however, also introduce new variables and destroy the structure of the formula. As a result, the alternative let-elimination typically makes problems much harder for the CDCL(LA) implementation than the standard let-elimination when the standard let-elimination does not cause an exponential blow-up. And even if the standard let-elimination causes an exponential blow-up, then it is still debatable whether the ruined formula structure of the alternative let-elimination is not the worse outcome.

It might be the worse outcome because we typically reduce the blowup caused by the standard let-elimination to a manageable size through some additional preprocessing steps. We start this process with some basic simplifications and an arithmetic flattening procedure, continue it with the elimination of the if-then-else operations, and end it with the CNF transformation.

Basic Simplifications

During our sharing process, we also apply the following series of equivalence preserving transformations to all subterms before we replace them with a shared version:

(ite t 1 false true) → (not t 1 ) (< t 1 t 2 ) → (not (>= t 1 t 2 )) (> t 1 t 2 ) → (not (<= t 1 t 2 ))
The above simplifications are all rather trivial because they only require a constant number of operator and pointer comparisons. 14 The simplifications do, however, reduce the overall formula size drastically; especially, when our let-elimination led to bad encodings. The effects of these basic simplifications can be divided into five categories: (i) they reduce a complex term to a truth value, (ii) they reduce a complex term to one of its arguments, (iii) they reduce the number of arguments of a complex term, (iv) they apply associativity to a complex term, or (v) they replace a complex operator (ite, <, or >) with a less complex combination of operators.

At the end of our sharing transformation, we also apply another basic simplification to our whole set of formulae F . We simply split every top-level conjunction in F , i.e., f ∈ F → F ∪ {t 1 , . . . , t m } \ f if f := (and t 1 . . . t m ). This transformation is equivalence preserving because the set F is interpreted as a conjunction of the formulae in the set.

Although the effects of these basic simplifications may seem trivial, they are also very beneficial for the other simplifications that we discuss in this section. For instance, they reduce the number of cases that we have to consider for the other simplifications. And since they are so cheap and beneficial, we also apply them for each new shared subterm constructed by our other simplifications.

Arithmetic Flattening

We also use the opportunity of the term sharing conversion to flatten our arithmetic terms into a more uniform structure. 15 We do so by applying the following equivalence preserving transformations to all processed arithmetic subterms:

(<= a 1 t 2 ) → (<= (+ a 1 ) t 2 ) (<= t 1 a 2 ) → (<= t 1 (+ a 2 )) (<= y 1 t 2 )
→ (<= (+ ( * 1 y 1 )) t 2 ) (+ q (+ q ) q) → (+ q q q) (+ q ( * a 1 t k+1 ) q ( * a 2 t k+1 ) q) → (+ q ( * (a 1 + a 2 ) t k+1 ) q q) (+ q a 1 q a 2 q) → (+ (a 1 + a 2 ) q q q) where q, q , q abbreviate sequences of term arguments, i.e., q := t 1 . . . t k , q := t 1 . . . t m , and q := t1 . . . tn , and infix operations are immediately evaluated, e.g., (a 1 /a 2 ) is evaluated (and replaced) by the actual result of a 1 /a 2 . As a result, our terms are now all constructed according to the following language: This means that we now only have one case for our arithmetic terms: they always are linear arithmetic sums, i.e., sums of linear arithmetic monomials, where each of the linear arithmetic monomials is either a constant monomial, i.e., (i) a constant number; (ii) a variable monomial, i.e., a constant number (coefficient) multiplied with a variable; or (iii) a complex monomial, i.e., a constant number (coefficient) multiplied with an if-then-else operation (complex term). Moreover, the linear arithmetic sums fulfill the following four conditions: (i) a linear arithmetic sum may only contain one constant monomial, (ii) if the sum contains a constant monomial, then it is the first argument of the sum, (ii) a linear arithmetic sum may only contain one variable monomial with the same variable, and (iii) a linear arithmetic sum may only contain one complex monomial with the same complex term. This has the effect that all linear arithmetic sums contain at most one constant monomial plus one monomial for every arithmetic variable plus one monomial for every arithmetic if-then-else expression. This again reduces the overall formula size drastically; especially, when our let-elimination led to bad encodings. Moreover, the above conditions allow us to treat any linear arithmetic sum as a map that maps each arithmetic variable and complex term to their coefficient value (if they actually appear in a monomial) or to zero. This is also one of the reason why we represent these sums internally as ordered tree maps with shared terms (variables or complex terms) as keys. The other reason is that this representation allows us to efficiently perform the arithmetic flattening. For instance, we can add/subtract two sums in linear time, add/subtract a single monomial to/from a sum in logarithmic time, and multiply a sum with a constant factor in linear time. 16Another added benefit of the arithmetic flattening is that it reduces the number of cases that we have to consider for our remaining preprocessing steps. For instance, we assign now with only five rules an arithmetic type to all arithmetic terms: And since the arithmetic flattening transformations are so cheap and beneficial, we also apply them for each new shared subterm constructed by our other simplifications.

x i : Integer if x i ∈ Zvars(F ) a i : Integer if a i ∈ Z ( * a i t i ) : Integer if a i ∈ Z

If-Then-Else Preprocessing

Our next series of preprocessing steps eliminates all if-then-else operations without increasing the formula size and if possible even reducing it. 17 One way of eliminating arithmetic if-then-else operators is to simply lift them over the comparators (<=, >=, =) so they become formula if-then-else operators. Then, we can eliminate the formula if-then-else operators based on the following equivalence derived from the semantic interpretation of the if-then-else operator:

(ite t 1 t 2 t 3 ) ≡ (and (or t 1 t 3 ) (or (not t 1 ) t 2 )). However, this semantic replacement for the formula if-then-else operation duplicates the condition t 1 . This means that replacing nested if-then-else operations in the condition t 1 causes an exponential blow-up in the formula size. Currently, this blow-up would not be a problem because our terms are shared, but the CNF transformation and the CDCL(LA) algorithm would again be impacted by it. Instead, we choose a standard if-thenelse-elimination that works similarly to the alternative let-elimination.

Standard If-Then-Else Elimination

As for the alternative let-elimination, we traverse all subterms of the form t := (ite t 1 t 2 t 3 ) in our set of formulae F . For each such subterm t, (i) we create a new propositional variable y i if the consequence t 2 and the alternative t 3 are formulae, (ii) we create a new integer variable y i if the consequence t 2 and the alternative t 3 are linear arithmetic sums and both of the Integer type, or (iii) we create a new rational variable y i if the consequence t 2 and the alternative t 3 are linear arithmetic sums and either is of the Rational type. Then, we replace all occurrences of t in F with y i and extend F by (or t 1 (= y i t 3 )) and (or (not t 1 ) (= y i t 2 )), i.e., F := F {t → y i } ∪ {(or t 1 (= y i t 3 )), (or (not t 1 ) (= y i t 2 ))}.

Since we base our standard if-then-else-elimination on the alternative let-elimination, it is no wonder that it also has the same disadvantages: (i) it has to introduce new variables; and (ii) it destroys the structure of the formula. Both are factors that decrease the efficiency of our CDCL(LA) implementation. For this reason, SPASS-SATT combines both approaches. During its optimal CNF transformation (Subsection 8.2.4) it decides for each formula if-then-else whether the semantic replacement or the standard if-then-else elimination leads to a better result. However, this is only an alternative for formula if-then-else operators. For arithmetic if-then-else operators, we first try to eliminate some if-thenelse operators with a series of additional preprocessing steps before we eliminate the remaining if-then-else operators with the standard elimination. These preprocessing steps include, if-then-else reconstruction, if-then-else simplifications over constants, boolean if-then-else compressions, selective if-then-else liftings, and bounding of arithmetic if-then-else operations.

If-Then-Else Reconstruction

We start our series of if-then-else preprocessing steps with the reconstruction of if-then-else operations. This is necessary because some input files in the SMT-LIB are already preprocessed and not always in a way that is advantageous for SPASS-SATT. For instance, the rings preprocessed benchmark family is equivalent to the rings benchmark family except that all if-then-else operations have been eliminated with standard if-then-else elimination [START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF]. However, the if-then-else operations in the rings benchmark family can be processed much more efficiently with different techniques. It is, therefore, a disadvantage that the standard if-then-else elimination was applied to the rings preprocessed benchmark family and we want to reverse it.

To do so, we check whether the set of formulae F contains any clauses that match the clauses added by the standard if-then-else elimination, i.e., whether there exists a set T i ⊆ {(or t i1 (= y i t i3 )), (or (not t i1 ) (= y i t i2 )), (or (= y i t i2 ) (= y i t i3 ))} with at least two elements (i.e., |T i | ≥ 2) that is contained in the set of formulae F (i.e., T i ⊆ F ). We then remove all such sets T i that we find from F and replace all occurrences of y i in F with t i := (ite t i1 t i2 t i3 ). This form of if-then-else reconstruction was not invented by us and it is implemented in several state-of-the-art SMT solvers, e.g., CVC4 [START_REF] Barrett | CVC4. In CAV[END_REF]. However, there does not yet exist any publication describing this process.

Constant If-Then-Else Simplifications

Our next step of if-then-else preprocessing handles so-called constant if-thenelse expressions (CITEs). A CITE cite is defined inductively according to the following language: cite ::= (+ cleaf ) | (+ number ( * number (ite formula cite cite ))) cleaf ::= number This means a CITE t is either (i) a leaf t := (+ a 0 ), which is a sum containing only a constant monomial, or (ii) a branch t := (+ a 0 ( * a 1 (ite t 1 t 2 t 3 ))), which is a sum containing only a constant monomial a 0 and a complex monomial ( * a 1 t ) such that the consequence t 2 and the alternative t 3 of the complex term t := (ite t 1 t 2 t 3 ) are also CITEs. Now our first preprocessing step for CITEs pushes all constant monomials and constant coefficients to the leaves:

(+ a 0 ( * a 1 (ite

t 1 t 2 t 3 ))) ↓ (+ ( * 1 (ite t 1 (+ a 0 ( * a 1 t 2 )) (+ a 0 ( * a 1 t 3 )))))
This means all occurrences of CITEs simplify and conform now to the following language:

cite ::= (+ cleaf ) | (+ ( * 1 (ite formula cite cite ))) cleaf ::= number This means a CITE t is now either (i) a constant leaf a 0 represented through the sum t := (+ a 0 ) containing only a 0 , or (ii) a branch t := (ite t 1 t 2 t 3 ) represented through t := (+ ( * 1 (ite t 1 t 2 t 3 ))), where t 2 and t 3 are also CITEs.

If-Then-Else Compression

Successful applications of the constant if-then-else simplifications, i.e., calls to csimp(o t 1 t 2 ) that actually simplify branches to true or false, often lead to nested if-then-else terms of the following form: t i ::= (ite t i t i+1 false) | (ite t i false t i+1 ) for i = 1, . . . , n -1 t n can be any formula term. (8.1) We are interested in terms of this form because we can efficiently eliminate if-then-else operations, where either the consequence or alternative is false. 18 For instance, if we look at the semantic interpretation of the term (ite t i t i+1 false), then we see that it evaluates only to true if (and t i t i+1 ) also evaluates to true. The term (ite t i t i+1 false) is, therefore, equivalent to the conjunction (and t i t i+1 ) and, symmetrically, (ite t i false t i+1 ) is equivalent to the conjunction (and (not t i ) t i+1 ). We can also apply this equivalence iteratively to the sequence of nested if-then-else terms t 1 , . . . , t n and combine the resulting conjunctions. The result is the equivalent conjunction (and t * 1 . . . t * n-1 t n ), where t * i := t i if t i := (ite t i t i+1 false) and t * i := (not t i ) if t i := (ite t i false t i+1 ).

In SPASS-SATT, we also transform nested if-then-else expressions of the form (8.1) into conjunctions. However, we perform the transformation more selectively than described above. We do so by traversing the subterms in our formulae f ∈ F in top-down order and by looking for any occurrence of a subterm t 1 of the form (8.1). Then we add all nested if-then-else terms t i to our sequence that also have the form (8.1) and that only appear once as a subterm in F . This means t n is either appearing more than once in F or it is not an if-then-else operation where the consequence or alternative is false. We then create a new propositional variable p j , replace all occurrences of t 1 in F with p j , and extend F by the equivalence (= p j (and t * 1 . . . t * n-1 t n )). We do this transformation so selectively in order to strengthen the connection of the shared subterms t n that have multiple occurrences in F . So the introduction of the new propositional variable p j actually strengthens the structure of our formulae instead of destroying it.

The above described technique is called if-then-else compression and it was first presented by Burch in [START_REF] Burch | Techniques for verifying superscalar microprocessors[END_REF]. However, Burch did not present it with SMT solving in mind but for an alternative application, verification of control circuits with the help of circuit simplifications. We are unaware of any publication describing if-then-else compression in the context of SMT solving although the SMT solver CVC4 has used this technique for quite some time [START_REF] Barrett | CVC4. In CAV[END_REF].

Lifting Shared Monomials

We have already shown how to simplify linear arithmetic atoms (o t 1 t 2 ), where t 1 and t 2 are constant if-then-else expressions (CITEs). However, other occurrences of CITEs, i.e., t := (ite t 1 t 2 t 3 ) in the linear arithmetic sum t := (+ q ( * a i t) q ) (where t 2 and t 3 are CITEs), are also easy to resolve. This is possible because we can optimize the standard if-thenelse elimination for CITEs with additional techniques, e.g., lifting of the least common multiple in constant if-then-else expressions and bounding of constant if-then-else expressions, which we explain later in this section.

Since CITEs are easier to handle than regular if-then-else expressions, we also want to simplify as many regular if-then-else expressions to constant if-then-else expressions as possible. This means we want to lift monomials shared by the consequence and the alternative of an if-then-else expression on top of the if-then-else expression; especially, if this leads to new constant if-then-else expressions. Formally, this means that we traverse the subterms in our formulae f ∈ F in bottom-up order and transform all subterms t := (ite t 1 (+ q q) (+ q q)) that we encounter into subterms (+ q (* 1 (ite t 1 (+ q ) (+ q)))), where q, q , and q are a series of monomials. 19In SPASS-SATT we call the above described technique lifting of shared monomials. We are unaware of any publication that describes lifting of shared monomials in the context of SMT solving. We know, however, that the SMT solver CVC4 has used this technique for quite some time [START_REF] Barrett | CVC4. In CAV[END_REF].

Bounding Constant If-Then-Else Expressions

We can actually combine the standard if-then-else elimination and the ifthen-else lifting for constant if-then-else expressions t. We simply create one new linear arithmetic variable x j for the whole constant if-then-else expression, replace all occurences of t with x j in F , and extend F by the formula f , which is equivalent to t except that all leaves a i of t are replaced by the equations (= x j a i ). The added formula if-then-else operations f are then handled by our optimal CNF transformation.

Although this combination reduces the number of newly created arithmetic variables, it still destroys a lot of structure that is now no longer accessible from the theory side. For instance, our theory solver does not know that the only legitimate values for x j correspond to the leaves {a 1 , . . . , a n } ∈ Q of t. Unfortunately, our arithmetic constraints are not expressive enough to efficiently limit x j to an arbitrary set of values {a 1 , . . . , a n } ∈ Q. Ho-wever, we can at least reduce the number of values x j can take to a finite interval [a min , a max ]. If the values {a 1 , . . . , a n } are actually integer values, then we can reduce the number of values x j can take even further until they correspond to a finite superset of {a 1 , . . . , a n }.

For the reduction, we preform the following steps: As our first step, we bound x j by a max := max{a 1 , . . . , a n } and a min := min{a 1 , . . . , a n }, i.e., we add the inequalities (<= x j a max ) and (>= x j a min ) to F . This means that x j can only only be assigned to values from the interval [a min , a max ]. As our second step, we turn the variable x j into an integer variable if the leaves {a 1 , . . . , a n } of t are all integer values. As a result, x j can only only be assigned to integer values from the interval [a min , a max ], which is a finite superset of {a 1 , . . . , a n }. Next, we try to shrink the distance between our leaves {a 1 , . . . , a n } and, therefore, the interval [a min , a max ] over which x j ranges. Note, however, that this step only works and is, therefore, only applied if the leaves {a 1 , . . . , a n } are all integer values. We perform the interval shrinking by computing the gcd a 0 := gcd{a 1 , . . . , a n } of our leaves and by replacing all occurrences of x j in F with ( * a 0 x j ). This means the equations (= x j a i ) in f become (= ( * a 0 x j ) a i ), which is equivalent to (= x j a i ), where a i := (a i /a 0 ) ∈ Z. The bounds (<= x j a max ) and (>= x j a min ) in F also change symmetrically, and are now equivalent to (<= x j a max ) and (>= x j a min ), where a max := max{a 1 , . . . , a n } and a min := min{a 1 , . . . , a n }. As a result, x j can only be assigned to the integer values from the interval [a min , a max ], which contains only 1 a 0 -th of the integer values that[a min , a max ] contained.

In SPASS-SATT we call the above described technique bounding constant if-then-else expressions. We are unaware of any publication that describes this bounding process in the context of SMT solving. We know, however, that the SMT solver CVC4 has used this technique for quite some time [START_REF] Barrett | CVC4. In CAV[END_REF].

Combination of If-Then-Else Preprocessing Steps

Bounding constant if-then-else expressions concludes the if-then-else preprocessing steps performed by SPASS-SATT. For a better overview, we summarize here the whole if-then-else preprocessing process: SPASS-SATT starts by reconstructing if-then-else operations that were naively eliminated from the input-file itself. Then, SPASS-SATT simplifies atoms over constant ifthen-else expressions. As its next step, SPASS-SATT compresses if-then-else expressions that are equivalent to conjunctions. After that, SPASS-SATT shifts its focus back to constant if-then-else expressions. It first tries to create more of them by lifting shared monomials over the if-then-else operations. The resulting constant if-then-else expressions are then bounded and eliminated by a combination of standard if-then-else elimination and ifthen-else lifting. Next, all remaining arithmetic if-then-else expressions are eliminated through standard if-then-else elimination. The formula if-thenelse expression are later handled as part of the optimal CNF transformation.

Other Simplifications

We have already presented several of SPASS-SATT's preprocessing techniques. Most of them were focused on reducing the formula size, on eliminating special operators, or on simplifying the overall formula structure. Our next series of preprocessing techniques focuses instead on optimizing the formula structure for our decision procedures.

Handling Variable Definitions

For instance, SPASS-SATT tries to remove some trivial variables, which we call defined variables. A defined variable y j has an equation f ∈ F that either assigns it to a truth value, e.g., f := (= y j true), a constant a i , e.g., f := (= y j a i ), another variable y i , e.g., f := (= y j y i ), a negated version of another variable (not y i ), e.g., f := (= y j (not y i )), or a negativ version of another variable (-y i ), e.g., f := (= y j (-y i )). This means it is quite easy to eliminate defined variables y j by simply replacing them with their assigned values.

Removing variables is beneficial to most of our decision procedures because their runtimes are dependent on the number of variables. We restrict ourselves to variables assigned to small terms because the runtimes of our decision procedures are also dependent on the number of monomials in each inequality and the number of literals in each clause.

Pseudo-Boolean Preprocessing

SPASS-SATT also optimizes the formula structure when it encounters socalled linear pseudo-boolean problems [START_REF] Peter | Boolean methods in operations research and related areas[END_REF]. Linear pseudo-boolean problems are ground and conjunctive linear arithmetic problems, where all arithmetic variables x j are actually pseudo-boolean variables, i.e., integer variables with bounds (>= x j 0) and (<= x j 1) in F ({(>= x j 0), (<= x j 1)} ⊆ F ). There actually exist many real world applications that can be encoded as linear pseudo-boolean problems [START_REF] Peter | Boolean methods in operations research and related areas[END_REF]. As a result, a specialized research community, the pseudo-boolean optimization community, emerged that focuses solely on these kinds of problems. And naturally, they have also developed their own specialized decision (and optimization) procedures for pseudoboolean problems.

Currently, the pseudo-boolean solvers with their specialized procedures still perform far better on pseudo-boolean problems than SPASS-SATT with its very general branch-and-bound approach. And this will not change until we have the time to reimplement some of their specialized procedures in SPASS-SATT. In the meantime, we have at least implemented a preprocessing technique that helps SPASS-SATT in closing the gap. To be more specific, our preprocessing technique turns some linear pseudo-boolean inequalities (i.e., inequalities containing just pseudo-boolean variables) into equivalent disjunctions and conjunctions of literals. For instance, the inequality (>= (+ t 1 . . . t n ) 1), where t i is either x i or (-1 x i ), is semantically equivalent to the disjunction (or l 1 . . . l n ), where l i is (>= x i 1) if t i = x i or l i is (not (>= x i 1)) if t i = (-1 x i ). Symmetrically, the inequality (>= (+ t 1 . . . t n ) n), where t i is either x i or (-1 x i ), is semantically equivalent to the conjunction (and l 1 . . . l n ), where l i is (>= x i 1) if t i = x i or l i is (not (>= x i 1)) if t i = (-1 x i ). SPASS-SATT recognizes all similar cases of linear pseudo-boolean inequalities that are equivalent to a single disjunction or a single conjunction with at most three literals. Moreover, SPASS-SATT replaces these inequalities with their equivalent disjunctions and conjunctions. We do so because the SAT solver seems to reason more efficiently over the disjunctions and conjunctions than our theory solver over the equivalent linear pseudo-boolean inequalities. However, this holds only for linear pseudo-boolean inequalities containing at most three variables. If SPASS-SATT replaces all linear pseudo-boolean inequalities that are equivalent to a single disjunction or a single conjunction, then it fails to solve some of the problems from the pigeonhole benchmark family.

The above described transformations go back to the NP-hardness proof of 0-1 programming [START_REF] Karp | Reducibility among combinatorial problems[END_REF]. Moreover, they are used by at least one other SMT solver (CVC4) [START_REF] Barrett | CVC4. In CAV[END_REF].

Small CNF Construction

The final preprocessing step of SPASS-SATT is the CNF transformation, i.e., the transformation of any general first-order formula into an equisatisfiable first-order formula in clause normal form (CNF). This step is necessary because our CDCL(LA) implementation, which combines most of SPASS-SATT's decision procedures, can only handle formulae in CNF. The standard CNF transformation works as follows [START_REF] Baaz | Normal form transformations[END_REF]. It first replaces all equivalences, implications, and if-then-else expressions with equivalent expressions consisting only of conjunctions, disjunctions, and negations, e.g., (= t 1 t 2 ) is equivalent to (or (and t 1 t 2 ) (and (not t 1 ) (not t 2 ))). Then, it pushes all negations down until there are only negations on top of the atoms (see also Section 8.2.2). Finally, it distributes all disjunctions inwards over conjunctions, i.e., it replaces all subterms (or t 1 (and t 2 t 3 )) with (and (or t 1 t 2 ) (or t 1 t 3 )). The advantage of the standard CNF transforma-tion is that it preserves not only equisatisfiability but also equivalence. This comes, however, at a cost: in the worst case, a formula grows during the standard CNF transformation by an exponential factor. The exponential growth is caused by nested applications of those parts of the transformation that duplicate subterms, i.e, the elimination of equivalences and if-then-else expressions and the distribution of disjunctions over conjunctions.

An alternative to the standard CNF transformation is the greedy renaming CNF transformation [START_REF] Grigori S Tseitin | On the complexity of derivation in propositional calculus[END_REF]. Renaming replaces a subterm t in a formula f with a new predicate p i , which is then defined as equivalent to t over the whole formula f by conjunctively adding the equivalence (= p i t) to f , i.e., f is transformed into (and f {t → p i } (= p i t)). The CNF transformation with greedy renaming now uses this technique to rename those subterms in f that would be duplicated by the standard CNF transformation. As a result, the renamed formula f contains no more nested subterms that would cause an exponential growth during the standard CNF transformation. Therefore, we can apply the standard CNF transformation after the remaining without encountering an exponential growth in the formula size.

Although the greedy renaming CNF transformation prevents the exponential growth of the standard CNF transformation, it also introduces four new problems. Firstly, the renaming adds new propositional variables and the runtime of CDCL(LA) is in the worst case exponentially proportional to the number of propositional variables. Secondly, the renaming also destroys connections in the original problem structure, which we could otherwise exploit in our decision procedures. Thirdly, the resulting formula is only equisatisfiable and not equivalent. 20 Finally, renaming sometimes creates a larger formula (or at least larger subformulae) then the standard CNF. But despite these new disadvantages, the greedy renaming CNF transformation typically performs much better in practice than the standard CNF transformation.

We can, however, get rid of the fourth disadvantage and lessen the first two disadvantages by renaming only those parts of the formula, where renaming leads to a smaller subformula than standard CNF. We call this CNF transformation the small CNF transformation and it was first presented by Nonnengart and Weidenbach [START_REF] Nonnengart | Computing small clause normal forms[END_REF]. The small CNF transformation always produces the smallest (sub)formula with respect to the standard CNF transformation and the greedy renaming CNF transformation. Moreover, the small CNF transformation always introduces at most as many new propositional variables as greedy renaming does. This means the small CNF transformation typically introduces less new propositional variables and destroys less structural connections. The implementation of the small CNF transformation was taken from the SPASS Workbench [START_REF]The SPASS Workbench[END_REF]. It replaced a previous implementation of the greedy renaming CNF transformation. Since we have both transformations available, we were also able to confirm that the small CNF transformation helps in practice. One example where the small CNF transformation performs much better than the greedy renaming CNF transformation is the convert benchmark family from the QF LIA category of the SMT-LIB. More details are provided in the preprocessing experiments at the end of this section.

Preprocessing Experiments

In this section, we described several preprocessing techniques that simplify input problems so they become easier to solve for SPASS-SATT. Techniques of particular interest are SPASS-SATT's preprocessing techniques for if-then-else expressions (constant if-then-else simplification, if-then-else compression, if-then-else reconstruction, shared monomial lifting, constant ifthen-else bounding), as well as SPASS-SATT's preprocessing techniques for variable definitions and pseudo-boolean inequalities. In order to measure the impact of these preprocessing techniques, we performed a series of benchmark experiments with SPASS-SATT. Our first two figures (Figure 8.12 & 8.13) examine the combined impact that SPASS-SATT's preprocessing techniques (for if-then-else expressions, defined variables, and pseudo-boolean inequalities) have on SPASS-SATT's performance on the QF LRA and QF LIA benchmarks. To this end, we compare the default version of SPASS-SATT with all preprocessing techniques turned on (horizontal axis) and the version of SPASS-SATT without the preprocessing techniques for if-then-else expressions, defined variables, and pseudo-boolean inequalities (vertical axis).

In Figure 8.12, we observe that our preprocessing techniques have only a minor impact on the QF LRA benchmarks. SPASS-SATT with our preprocessing techniques solves only two more QF LRA benchmark instances (both from the latendresse benchmark family). However, we can at least conclude that these additions are independent of any cluster fluctuations since they occur far enough away from our timeout limit. Otherwise, we observe that SPASS-SATT with our preprocessing instances is significantly faster (i.e., it is more than twice as fast) as often as it is significantly slower (i.e., it is more than twice as slow). We were unable to discern a clear pattern that would allow us to predict when our preprocessing techniques are advantageous on the QF LRA benchmarks.

In Figure 8.13, we observe that our preprocessing techniques have a major impact on the QF LIA benchmarks. SPASS-SATT with our preprocessing techniques solves 1693 additional QF LIA benchmark instances and is significantly faster (i.e., it is more than twice as fast) on 3146 QF LIA benchmark instances than SPASS-SATT without our preprocessing techniques. Moreover, instances where our preprocessing techniques cause a significant slowdown occur only rarely. Also in contrast to the QF LRA results, we can clearly determine four benchmark families, on which our preprocessing techniques make this positive impact. These families are nec smt, rings, rings preprocessed, and pb2010; we are going to analyze them in more detail as part of this section.

The nec smt Benchmark Family

The instances from the nec smt benchmark family were generated by the SMT-based boolean model checking engine of F-Soft, which is a software verification platform [START_REF] Ivancic | F-soft: Software verification platform[END_REF]. All instances from nec smt have in common that they contain many nested if-then-else and let expressions. As a result of this nesting, we cannot just remove the let operations or else we are creating formulas that are too big to be handled within reasonable memory and time limits. We can, however, handle them if we additionally simplify the nested if-then-else expressions. The two preprocessing techniques that make this possible are our constant if-then-else simplifications and the boolean if-thenelse compression (Subsection 8.2.3).

In Figure 8.14, we see that SPASS-SATT without our preprocessing techniques solves only 1422 out of the 2800 nec smt benchmark instances and is by far slower on the instances it can solve. In Figures 8.15 & 8.16, we see that SPASS-SATT performs even worse if we only perform one of our two preprocessing techniques. 21 However, SPASS-SATT with both constant if-then-else simplifications and the boolean if-then-else compression solves 2782 out of the 2800 benchmark instances. This means that only the combination of the two techniques produces an actually beneficial result. Moreover, the two preprocessing techniques simplify the input problems so much that SPASS-SATT does not need branch-and-bound or one of its extensions to solve these instances. SPASS-SATT's simplex implementation on its own is sufficient enough and detects for all solved instances either a rational conflict or a rational solution that is also an integer solution. The rings preprocessed benchmark family is equivalent to the rings benchmark family except that all if-then-else operations were eliminated with standard if-then-else elimination [START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF]. It was constructed in order to prevent the previously explained preprocessing trick for the rings benchmark family [START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF]. However, we can use the same trick that we used for the rings benchmark family also for the rings preprocessed benchmark family if we just use our if-then-else reconstruction technique to reverses the standard if-then-else elimination (Subsection 8.2.3). In Figure 8.20, we see that SPASS-SATT without if-then-else reconstruction performs as poorly on the rings preprocessed benchmark family as it performs on the rings benchmark family without any preprocessing techniques. However, SPASS-SATT with if-then-else reconstruction solves again all of the 294 benchmark instances in no time. The pb2010 benchmark family is a set of industrial problems taken from the pseudo-boolean competition 2010. Naturally, all of the contained problems are pseudo-boolean problems and, therefore, rather difficult for solvers without specialized decision procedures for pseudo-boolean problems. This includes SPASS-SATT with its very general branch-and-bound approach. In fact, SPASS-SATT does not use/need the branch-and-bound approach for the pseudo-boolean problems it can solve within the time limit. Instead, SPASS-SATT uses its preprocessing technique for pseudo-boolean inequalities to shift the difficulty of the problem instances from arithmetic to SAT solving. This allows SPASS-SATT to solve 22 more benchmark instances from the 81 instances in the pb2010 benchmark family (see Figure 8

Handling Variable Definitions

Another one of our more specialized preprocessing techniques is the elimination of defined variables. In Figures 8.24 & 8.25, we examine the impact that the elimination of defined variables has on SPASS-SATT's performance on the QF LRA and QF LIA benchmarks. In these figures, we can observe that the elimination of defined variables has only a minor impact on SPASS-SATT's performance. However, we can also see that there are at least a few problems from both QF LRA and QF LIA that we can only solve thanks to the elimination of defined variables. These problems are 2 problems from the latendresse benchmark family and 2 problems from the bofill-scheduling benchmark family. The other additionally solved benchmark instances are solved so close to the timeout that they probably occurred due to performance fluctuations of our cluster. As explained before, SPASS-SATT uses a small CNF transformation instead of the commonly used greedy renaming CNF transformation. Naturally, this also has an impact on SPASS-SATT's performance on the SMT-LIB benchmarks. In Figures 8.24 & 8.25, we examine the impact that the small CNF transformation has on SPASS-SATT's performance on the QF LRA and QF LIA benchmarks compared to the greedy renaming CNF transformation. In these figures, we see that the small CNF transformation leads to overall better results than the greedy renaming CNF transformation. In spite of that, there are many instances where the greedy renaming CNF transformation performs better. We suspect, however, that most of the positive and negative changes to the results are not because one transformation produces a better clause set, but because our new transformation orders the clauses and literals by accident differently than our old transformation.

There is, however, one benchmark family in the QF LIA division where the small CNF transformation produces a beneficial result because it actually produces a better clause set and not just a better clause order. This benchmark family is called convert and Figure 8.26 examines the impact that the small CNF transformation has on SPASS-SATT's performance on the convert benchmark family compared to the greedy renaming CNF transformation. In this figure, we can see that SPASS-SATT with the small CNF transformation solves all 319 convert instances in a couple of seconds, and SPASS-SATT with the greedy renaming CNF transformation can only solve 218 out of the 319 instances. The reason is that most formulas in the convert family look abstractly as follows:

F := (and . . . f 1 . . . f n . . .) , where f i := (or (= t i1 t i2 ) (not (= t i3 t i4 ))) . This formula structure is problematic for our implementation of the greedy renaming CNF transformation because of two implementation choices. Firstly, SPASS-SATT splits all occurring arithmetic equalities into two inequalities in order to handle negated arithmetic equalities. Therefore, SPASS-SATT transforms the subformulas f i into f i := (or (and (<= t i1 t i2 ) (>= t i1 t i2 )) (< t i3 t i4 ) (> t i3 t i4 )) . Secondly, SPASS-SATT applies the renamings in the greedy renaming CNF transformation polarity dependent. This means the greedy renaming CNF transformation transforms the subformulas f i into f g i := (and (or p i (< t i3 t i4 ) (> t i3 t i4 )) (or (not p i ) (<= t i1 t i2 )) (or (not p i ) (>= t i1 t i2 ))) , where the p i are propositional variables freshly introduced for the renamings. For comparison, SPASS-SATT with the small CNF transformation would not apply any renamings to the subformulas f i , but would apply the standard CNF transformation because it generates a smaller subformula. f m i := (and (or (<= t i1 t i2 ) (< t i3 t i4 ) (> t i3 t i4 )) (or (>= t i1 t i2 ) (< t i3 t i4 ) (> t i3 t i4 ))) . Naturally, f m i is better than f g i because it is smaller and does not contain any new propositional variables. But due to the polarity dependent renaming in f g i , f m i is also more expressive, i.e., there are potential unit propagations, which can be performed with f m i but that cannot be simulated with f g i . 22 We could fix this by performing the renamings in the greedy renaming CNF transformation polarity independent, but this would also mean that the resulting formula gets bigger. 22 The only unit propagations, which can be performed with f g i but that cannot be simulate with f m i , propagate pi or (not pi), so literals that do not even occur in f m i .

Conclusion

We started this thesis by extending the incomplete CutSat calculus by Jovanović and de Moura [START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. The result was our CutSat++ calculus (Chapter 3), which is the first sound, complete, and terminating calculus for linear integer problems based on the model assumption and conflict learning approach motivated by CDCL style SAT solving. In addition to the techniques derived from CDCL style SAT solving, CutSat++ also uses a lazy quantifier elimination scheme with which it handles unbounded problems.

There is a reasonable gap between the CutSat++ calculus and an efficient implementation of it. Firstly, we found no intuitive way with which we could extend CutSat++ to linear mixed arithmetic. Secondly, many techniques that make CDCL-based SAT solving efficient are not straightforward to incorporate into CutSat++ due to the combination with the lazy quantifier elimination scheme. For example, the efficiency of CDCL-based SAT relies on a heuristic for picking decision variables that is refined dynamically during the runtime. For CutSat++, a similarly dynamic decision heuristic seems to be impossible because the a priori fixed variable order, which is used to guarantee termination, also has a lot of influence on the way CutSat++ picks variables for decisions. We assume that this becomes a problem for an efficient implementation because of observations of other calculi with similar limitations. For instance, we have observed a difference in efficiency between CDCL-based SAT solving and superposition-based SAT solving, which is also based on model assumptions and a fixed a priori ordering. Both calculi are guaranteed to terminate on the ground fragment of first-order logic. However, it seems that the lazy, dynamic and problem driven way CDCL-based SAT solving develops the variable order [START_REF] Weidenbach | Automated reasoning building blocks[END_REF] is more efficient, in general. Currently, it is still an open problem whether Cut-Sat++ can be further refined so that the ordering becomes dynamic and the calculus still guarantees termination. Otherwise, we hope that heuristics motivated by quantifier elimination procedures may yield further insights in how to select the best possible a priori fixed variable order for a given input problem.

Alongside CutSat++, we developed additional ideas for handling unbounded problems. These ideas are all based on a deeper analysis of unbounded problems. For instance, unbounded problems can be intuitively partitioned into two categories: absolutely unbounded and partially bounded problems. Partially bounded problems contain, in fact, all of the hard unbounded problems and absolutely unbounded problems are trivial to detect and to solve with the right methods. This led us to the development of the fast cube tests (Chapter 4), which can detect and solve absolutely unbounded problems in polynomial time.

Our two fast cube test are named the largest cube test and the unit cube test. In contrast to many complete methods that search along the problem surface for a solution, these tests use cubes to explore the interior of the problem. The largest cube test finds a cube with maximum edge length contained in the rational solutions of the input problem, determines its rational valued center, and rounds it to a potential mixed/integer solution. The unit cube test determines instead whether the the rational solutions of the input problem contain a cube with edge length one, which is the minimal edge length that always guarantees that a cube contains a mixed/integer solution.

The tests are especially efficient on constraint systems with a large number of integer solutions, e.g., those that are absolutely unbounded. Inside the SMT-LIB benchmarks, we have found almost one thousand problem instances that are absolutely unbounded. Benchmark evaluations confirm that our tests are superior on these instances compared to several state-of-the-art SMT solvers (Section 4.5). As a result, other SMT solvers (e.g., MATHSAT and Z3) now also employ our unit cube tests [START_REF] Bjorner | Theorem recycling for theorem proving[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF].

Conceptually, it is also possible to generalize our tests so they search for any d-norm ball (where d is fixed) and not just cubes. However, the resulting d-norm ball tests are no longer guaranteed to be linear programs because they require inequality bounds that may contain roots. This means that we cannot directly solve general d-norm ball tests with our linear arithmetic solver but at best over-approximate them.

One major obstacle for a wider application of our cube tests are equalities. To resolve this obstacle, we developed several techniques for the investigation and removal of equalities (Chapter 5). These methods also have many additional applications besides simplifications for our cube tests. For instance, as a short cut in quantifier elimination procedures (Section 5.6) and as part of the Nelson-Oppen combination of theories (Section 5.4). But the most important application with regard to the main theme of this thesis is their capability to efficiently detect all (un)bounded directions (Section 5.5).

3 . 11 A

 311 2.10 A selection of two-dimensional d-norm balls; all with the same radius and center point . . . . . . . . . . . . . . . . . . . . . 3.1 The decision and propagation rules of CutSat g . . . . . . . 3.2 The (guarded) conflict resolution rules of CutSat g . . . . . 3.3 Rule system that derives tightly propagating inequalities [87] 3.4 The Learn and Forget rules of CutSat g . . . . . . . . . . . . 3.5 The rules of CutSat g that lead to end states . . . . . . . . . 3.6 The Slack-Intro rule of CutSat g . . . . . . . . . . . . . . . . 3.7 The strong conflict resolution rules by [87] . . . . . . . . . . . 3.8 An algorithm that combines all divisibility constraints containing x j until only one such constraint remains . . . . . . . 3.9 Our unguarded conflict resolution rules . . . . . . . . . . . . . 3.10 A CutSat++ run for Example 3.5.8 depicted as a transition of states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XIII CutSat++ run for Example 3.5.9 depicted as a transition of states . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.12 A CutSat++ run for Example 3.5.10 . . . . . . . . . . . . . 3.13 The first part of a CutSat++ run for Example 3.5.11. Continued in Figure 3.14 . . . . . . . . . . . . . . . . . . . . . . . 3.14 The second part of a CutSat++ run for Example 3.5.11. Continued in Figure 3.15 . . . . . . . . . . . . . . . . . . . . . 3.15 The third part of a CutSat++ run for Example 3.5.11. Continued in Figure 3.16 . . . . . . . . . . . . . . . . . . . . . . . 3.16 The fourth part of a CutSat++ run for Example 3.5.11 . . 3.17 A CutSat++ run for Example 3.5.12 . . . . . . . . . . . . . 4.1 A square (two-dimensional cube) fitting into an inequality a T i x ≤ b i and the cube's maximum a T i x * for the objective a T i x 4.2 The vertices of an arbitrary axis-parallel square (two-dimensional cube with edge length e and center z) . . . . . . . . . . . . . 4.3 The transformed polyhedron Ax ≤ b for edge length 1 together with the original polyhedron Ax ≤ b . . . . . . . . . . . . 4.4 The largest cube inside a polyhedron, its center point, and a closest integer point to the center . . . . . . . . . . . . . . . . 4.5 An absolutely unbounded polyhedron containing cubes for every edge length e > 0. . . . . . . . . . . . . . . . . . . . . . 4.6 A polyhedron for which the cube tests fail . . . . . . . . . . . 4.7 A unit cube inside a polyhedron, its center point, and a closest integer point to the center . . . . . . . . . . . . . . . . . . . . 4.8 Experimental Results: SMT solvers . . . . . . . . . . . . . . . 4.9 Experimental Results: MILP solvers . . . . . . . . . . . . . . 5.1 EqBasis computes an equality basis . . . . . . . . . . . . . . 5.2 The main function for computing an equality basis in tableau representation . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Initialization function for computing an equality basis in tableau representation . . . . . . . . . . . . . . . . . . . . . . . 5.4 Supporting function for computing an equality basis in tableau representation . . . . . . . . . . . . . . . . . . . . . . . 6.1 A partially bounded system . . . . . . . . . . . . . . . . . . . 6.2 The (double-)bounded part of Figure 6.1; v j := (1, 1) T is an orthogonal direction to the bounding row vectors . . . . . . . 6.3 The unbounded part of Figure 6.1 . . . . . . . . . . . . . . . 6.4 ExtendMEH() extends a Mixed-Echelon-Hermite normal form by one inequality. All algorithms used in the transformation are based on algorithms from [128]. . . . . . . . . . . . . . . . XIV 6.

Figure 1 . 1 : 2 x 1 Figure 1 . 2 : 2 x 1 Figure 1 . 3 :

 1121122113 Figure 1.1: The rational solutions of our linear arithmetic example problem

Figure 1 2 x 1 Figure 1 . 5 : 2 x 1 Figure 1 . 6 :

 121152116 Figure 1.4: A partially bounded system; the directions h = (-1, 1) T and -h are the only bounded directions in the example

  n 1 := | Qvars(C)| variables x 1 , . . . , x n 1 are rational variables and the remaining n 2 := | Qvars(C)| variables x n 1 +1 , . . . , x n are integer variables, where n = n 1 + n 2 := | vars(C)|.

  For the remainder of this thesis, we abbreviate with b δ i the strict version of a given bound b

10 a

 10 lj := a lj a ij ; 11 end 12 N := {z 1 , . . . , z n }; B := {y 1 , . . . , y m }; (Az = y) := (A z = y );

Figure 2 . 3 :

 23 Figure 2.3: A conflict extraction function for the simplex algorithm[START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]

  e., (Ax ≤ b) ∪ (Dx ≤ c) follows (Ax ≤ b), or the problems are decrementally connected , i.e., (Ax ≤ b) follows (Ax ≤ b) ∪ (Dx ≤ c).

Figure 2 . 5 :

 25 Figure 2.5: A simplex backtrack function[START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]

Figure 2 . 9 :

 29 Figure 2.9: Two series of problems for which branch-and-bound diverges.

  The resulting model assumption [[P, ¬Q]] falsifies the clause ¬P ∨ Q. Then a resolution step between this clause and the propagating clause ¬P ∨ ¬Q yields the clause ¬P . This clause repairs the model via backtracking into [[¬P ]] and after one propagation into [[¬P, Q]], which is an overall model of the three clauses.

  The produced bounds are stored in a sequence M = [[γ 1 , . . . , γ n ]] and they describe a partial model for C. Out of convenience, we use [[M, γ]] and [[M 1 , M 2 ]] to denote the concatenation of a bound γ at the end of M , and M 2 at the end of M 1 , respectively. Moreover, we denote by [[]] the empty sequence.

  as S = M, C and call it a search state. Otherwise, we call M, C, I a conflict state. The start state for a problem C is the search state [[]], C . Therefore, the start states of CutSat++ are all search states [[]], C , where C is a constraint system.

Figure 3 . 2 :

 32 Figure 3.2: The (guarded) conflict resolution rules of CutSat g

Figure 3 . 4 :

 34 Figure 3.4: The Learn and Forget rules of CutSat g

  where d = gcd(a 1j d 2j , a 2j d 1j ), and c 1j and c 2j are integers such that c 1j a 1j d 2j + c 2j a 2j d 1j = d [44, 87].

Figure 3 . 7 :

 37 Figure 3.7: The strong conflict resolution rules by[START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]

Example 3 . 3 . 6 .

 336 Let d i be the sequence with d 0 := 2 and d

Lemma 3 . 4 . 8 (

 348 CombDivs Equivalence). Let C be a set of LIA constraints. Let C be the set of LIA constraints we receive from CombDivs(x j , C ). Then C ≡ C .

Algorithm 10 :

 10 CombDivs(x j , C ) Input : A variable x j and a set of LIA constraints C Output: A set of LIA constraints C such that C ≡ C and there exists exactly one divisibility constraint d

  M ) holds and d | a j x j +p j is the only divisibility constraint in C with x j as top variable. 2. Let x j be an unguarded variable. Let ∈ {≤, ≥}. Then we only allow decisions γ = x j b j if: (a) for every constraint I ∈ C with x j = top(I) all occurring variables x i = x j are fixed (b) there exists no I ∈ C where x j = top(I) and I is a conflict in [[M, γ]]

Figure 3 .

 3 Figure 3.10: A CutSat++ run for Example 3.5.8 depicted as a transition of states

Figure 3 .

 3 Figure 3.11: A CutSat++ run for Example 3.5.9 depicted as a transition of states

Figure 3 .

 3 Figure 3.12: A CutSat++ run for Example 3.5.10

  , 3.14, 3.15, and 3.16.

Figure 3 . 13 :

 313 Figure 3.13: The first part of a CutSat++ run for Example 3.5.11. Continued in Figure 3.14

Figure 3 . 14 :

 314 Figure 3.14: The second part of a CutSat++ run for Example 3.5.11. Continued in Figure 3.15

CS

Figure 3 . 1 , x 1 - 1 ≤ 0 I 2 , -x 2 ≤ 0 I 3 , 6 | 4y + x 1 J}

 3111020361 Figure 3.17: A CutSat++ run for Example 3.5.12

Lemma 3 . 6 . 1 (

 361 Resolvent Stability). Let S = M, C be a state reachable by CutSat++ from the start state [[]], C 0 and let S = M , C be a state reachable by CutSat++ from S. Let C contain an unguarded resolvent R for (x j , C ). Then C contains also an unguarded resolvent R for (x j , C ).

  5.3, the new problem C = C ∪ R contains an unguarded resolvent R for (x i , C * ). Therefore, | woSR(x i , C )| > | woSR(x i , C)|. By Definition 3.5.2, it holds for all x k ∈ vars(R) that x k ≺ x i , which implies that k < i. Hence, Resolve-Weak-Cooper has not introduced new potential conflicting cores (x j , C * * ) with j ≥ i and | cores(x j , C )| ≥ | cores(x j , C)| for all j ≥ i. By Lemma 3.6.1, | woSR(x j , C )| ≥ | woSR(x j , C)| for all j > i. Therefore, the weight decreases after an application of Resolve-Weak-Cooper, i.e., W(S ) > lex W(S).

Lemma 3 . 6 . 8 (

 368 Eager Top-Level Stability). If S is an eager top-level propagated state (Definition 3.6.7), then any successor state S = M, C, I reachable by CutSat++ is eager top-level propagated.

2 .

 2 Let the applied rule be Decide. Then S = M , C and S = [[M , x i b i ]],

Figure 4 Figure 4 . 2 : 2 x 1 Figure 4 . 3 :

 4422143 Figure 4.1: A square (two-dimensional cube) fitting into an inequality a T i x ≤ b i and the cube's maximum a T i x * for the objective a T i x Figure 4.2: The vertices of an arbitrary axis-parallel square (two-dimensional cube with edge length e and center z)

  the cube fits into Ax ≤ b if and only if it fits into all inequalities a T i x ≤ b i : ∀i ∈ {1, . . . , m}. ∀x ∈ C e (z). a T i x ≤ b i . We can express this by m optimization problems: ∀i ∈ {1, . . . , m}. max{a T i x : x ∈ C e (z)} ≤ b i and, after applying Proposition 4.2.1, by the following m inequalities: ∀i ∈ {1, . . . , m}. a T i z ≤ b i -e 2 a i 1 . Hence, the linear cube transformation transforms the polyhedron Ax ≤ b into the polyhedron Ax ≤ b , where b i = b i -e 2 a i 1 , and C e (z) fits into Ax ≤ b if Az ≤ b . Corollary 4.2.2 (Linear Cube Transformation). Let C e (z) be a cube and Ax ≤ b be a polyhedron. C e (z) ⊆ Q δ (Ax ≤ b) if and only if Az ≤ b , where b i = b i -e 2 a i 1 .

Figure 4 . 4 : 2 x 1 Figure 4 . 5 :

 442145 Figure 4.4: The largest cube inside a polyhedron, its center point, and a closest integer point to the center

Figure 4 . 6 : 2 x 1 Figure 4 . 7 :

 462147 Figure 4.6: A polyhedron for which the cube tests fail

Figure 4 . 9 :

 49 Figure 4.9: Experimental Results: MILP solvers

y i y k b

  i . Now, we reformulate y T b = 0 analogously and get: b k = -m i=1,i =k y i y k b i . Thus, a T k x = b k is an equality contained in the original inequalities Ax ≤ b. If the maximum objective value is negative, Ax ≤ b is unsatisfiable and contains no cube with edge length e > 0. Since Q δ (Ax ≤ b) is now empty, Ax ≤ b contains all equalities.

  basis. To this end, EqBasis computes in each iteration one system of inequalities Az ≤ b and one system of equalities y -Dz = c such that A x ≤ b is equivalent to (y -Dz = c) ∪ (Az ≤ b). While the variables z are completely defined by the inequalities Az ≤ b, the equalities y -Dz = c extend any assignment from the variables z to the variables y. Initially, z is just x, y -Dz = c is empty, and Az ≤ b is just A x ≤ b . In every iteration l of the while loop, EqBasis eliminates one equality a T i z = b i from Az ≤ b and adds it to y -Dz = c. EqBasis finds this equality based on the techniques we presented in the Lemmas 5.2.2 & 5.2.3 (line 7). If the current system of inequalities Az ≤ b implies no equality, then EqBasis is done and returns the current system of equalities y -Dz = c. Otherwise, EqBasis turns the found equality a T i z = b i into a substitution σ := {z k → b i a ik -n j=1,j =k a ij a ik z j } (line 11) and applies it to Az ≤ b (line 15

Lemma 5 . 2 . 5 (

 525 Equivalent Substitution). Let y -Dz = c be a satisfiable system of equalities. Let Ax ≤ b and A * x ≤ b * be two systems of inequalities, both implying the equalities in y -Dz = c. Let A z ≤ b := (Ax ≤ b)σ D,c y,z and A * * z ≤ b * * := (A * x ≤ b * )σ D,c y,z . Then A z ≤ b is equivalent to A * * z ≤ b * * if Ax ≤ b is equivalent to A * x ≤ b * . Proof. Let Ax ≤ b be equivalent to A * x ≤ b * . Suppose to the contrary that A z ≤ b is not equivalent to A * * z ≤ b * * . This means that there exists a v ∈ Q nz δ such that either A v ≤ b is true and A * * v ≤ b * * is false, or A v ≤ b is false and A * * v ≤ b * * is true. Without loss of generality we select the first case that A v ≤ b is true and A * * v ≤ b * * is false. We now extend this solution by u ∈ Q ny δ , where u i := c i + d T i v, so (A v ≤ b ) ∪ (u -Dv = c) is true. Based on the definition of the substitution σ D,c y,z and n y recursive applications of Lemma 5.2.4, the four constraint systems Ax ≤ b, A * x ≤ b * , (A z ≤ b ) ∪ (y -Dz = c), and (A * * z ≤ b * * ) ∪ (y -Dz = c) are equivalent. Therefore, (A * * v ≤ b * * ) ∪ (u -Dv = c) is true, which means that A * * v ≤ b * * is also true. The latter contradicts our initial assumptions. Now we can also prove what we have already explained at the beginning of this section. The equality h T x = g is implied by Ax ≤ b if and only if y -Dz = c is an equality basis and (h T x = g)σ D,c

Lemma 5 . 2 . 7 (

 527 Equality Basis). Let A x ≤ b be a satisfiable system of inequalities. Let y -Dz = c be the output of EqBasis(A x ≤ b ). Then y -Dz = c is an equality basis of A x ≤ b . Proof. Let Az ≤ b be the result of applying σ D,c y,z to A x ≤ b . Then the condition in line 7 of EqBasis and y -Dz = c being the output of EqBasis(A x ≤ b ) guarantee us that Az ≤ b implies no equalities.

  Figure 5.3: Initialization function for computing an equality basis in tableau representation

Figure 5 . 4 :

 54 Figure 5.4: Supporting function for computing an equality basis in tableau representation

2 x 1 Figure 6 . 3 :

 2163 Figure 6.1: A partially bounded system

4 s 6 V 7 HFigure 6 . 10 :

 467610 Figure 6.10: ReduceRightI(H , V , p)

Figure 6 . 11 :

 611 Figure 6.11: SPASS-IQ compared to SMT solvers on the SlackedQFLIA and FlippedQFLIA benchmark families

Figure 6 . 13 :

 613 Figure 6.13: SPASS-IQ compared to various MILP solvers on the SlackedQFLIA and RandomUnbd benchmark families

Figure 7 . 1 :

 71 Figure 7.1: Results for SPASS-IQ's pivoting strategies on the SMT-LIB benchmark division QF LRA

Figure 7 . 4 :

 74 Figure 7.4: Results for SPASS-IQ's pivoting strategies on the SMT-LIB benchmark division QF LIA without switching to Bland's rule after finitely many pivots

10 -2 10 - 3 Figure 7 . 6 : 3 Figure 7 . 7 : 1 Figure 7 . 8 :

 10376377178 Figure 7.6: Greedy vs. lazy conflict detection on the QF LIA SMT-LIB benchmarks

10 -2 10 -Figure 7 . 10 :Figure 7 . 11 :Figure 7 . 12 :

 10710711712 Figure 7.10: Backtracking via recalculation (HardBacktrack()) vs. backtracking via backup (Backtrack()) on the QF LRA SMT-LIB benchmarks

10 -2 10 -

 10

3 FLINTFigure 7 . 14 :

 3714 Figure 7.14: FLINT vs. GMP on the QF LIA SMT-LIB benchmarks

Figure 7 . 15 :

 715 Figure 7.15: Data structures for variables and bounds

3 Figure 7 . 23 :

 3723 Figure 7.23: With(out) branch-and-bound extensions on the QF LRA SMT-LIB benchmarks

10 -2 10 - 3 Figure 7 . 25 :Figure 7 .

 1037257 Figure 7.25: With(out) simple rounding on the QF LIA SMT-LIB benchmarks

Figure 7 . 26 :

 726 Figure 7.26: Results for different bound propagation settings on the SMT-LIB benchmark division QF LIA

3 Figure 7 . 27 :

 3727 Figure 7.27: With(out) unit cube test on the QF LIA SMT-LIB benchmarks

10 -2 10 - 1 WithFigure 7 . 28 :

 101728 Figure 7.28: With(out) bounding transformations on the QF LIA SMT-LIB benchmarks

Figure 8 . 1 :

 81 Figure 8.1: A control flow graph that describes the four phases of CDCL(LA) in SPASS-SATT.

Figure 8 . 5 :

 85 Figure 8.5: A control flow graph that describes SPASS-SATT's solver interactions during the conflict phase.

3 Figure 8 . 6 :

 386 Figure 8.6: With(out) Decision Recommendation on the QF LRA SMT-LIB benchmarks

3 Figure 8 . 7 :

 387 Figure 8.7: With(out) Decision Recommendation on the QF LIA SMT-LIB benchmarks

10 -2 10 - 3 Figure 8 . 8 :

 10388 Figure 8.8: With(out) Unate Propagation on the QF LRA SMT-LIB benchmarks

10 -2 10 - 3 Figure 8 . 9 :

 10389 Figure 8.9: With(out) Unate Propagation on the QF LIA SMT-LIB benchmarks

Figure 8 . 10 :

 810 Figure 8.10: Results for different bound refinement settings on the SMT-LIB benchmark division QF LRA

  and

Figure 8 . 11 :

 811 Figure 8.11: Results for different bound refinement settings on the SMT-LIB benchmark division QF LIA

  formula ::= (and formula * ) | (or formula * ) | (not formula ) | (=> formula formula ) | (= formula formula + ) | (ite formula formula formula ) | (let let defs formula ) | let var | prop var | true | false | (<= la term la term ) | (>= la term la term ) | (< la term la term ) | (> la term la term ) | (= la term la term ) la term ::= (+ la term la term + ) | (la term la term ) | (la term ) | ( * la term la term + ) | (/ la term la term ) | number | let var | la var | (ite formula la term la term ) | (let let defs la term ) let defs ::= ( let def + ) let def ::= ( let var formula ) | ( let var la term ) prop var ::= identifier let var ::= identifier la var ::= identifier number ::= integer | rational integer ::

  formula ::= (and formula * ) | (or formula * ) | (not formula ) | (=> formula formula ) | (= formula formula + ) | (ite formula formula formula ) | prop var | true | false | (<= la sum la sum ) | (>= la sum la sum ) | (= la sum la sum ) la sum ::= (+ la monom + ) la monom ::= ( * number la var ) | ( * number la ite ) | number la ite ::= (ite formula la sum la sum ) la var ::= int var | rat var number ::= integer | rational

10 -2 10 - 3 Figure 8 . 12 :

 103812 Figure 8.12: With(out) preprocessing techniques on the QF LRA SMT-LIB benchmarks

3 Figure 8 . 13 :

 3813 Figure 8.13: With(out) preprocessing techniques on the QF LIA SMT-LIB benchmarks

10 -2 10 - 3 Figure 8 . 14 : 3 Figure 8 . 15 : 1 3 Figure 8 . 17 : 3 Figure 8 . 18 : 1 With

 10381438151381738181 Figure 8.14: With(out) extensive preprocessing on the QF LIA nec smt benchmark family

3 Figure 8 . 20 :

 3820 Figure 8.20: With(out) if-then-else (ITE) reconstruction on the QF LIA rings preprocessed benchmark family

3 Figure 8 . 21 :

 3821 Figure 8.21: With(out) pseudo-boolean preprocessing on the QF LIA pb2010 benchmark family

3 Figure 8 . 22 : 3 Figure 8 . 23 :

 38223823 Figure 8.22: With(out) defined variable simplifications on the QF LRA SMT-LIB benchmarks

3 Figure 8 . 24 :

 3824 Figure 8.24: With(out) small CNF on the QF LRA SMT-LIB benchmarks

3 Figure 8 . 25 : 3 Figure 8 . 26 :

 38253826 Figure 8.25: With(out) small CNF on the QF LIA SMT-LIB benchmarks

  .10 ReduceRightI(H , V , p) . . . . . . . . . . . . . . . . . . . . . 6.11 SPASS-IQ compared to SMT solvers on the SlackedQFLIA and FlippedQFLIA benchmark families . . . . . . . . . . . . . 6.12 SPASS-IQ compared to various SMT solvers on the RandomUnbd and FlippedRandomUnbd benchmark families . . . . . . . . . 6.13 SPASS-IQ compared to various MILP solvers on the SlackedQFLIA and RandomUnbd benchmark families . . . . . . . . . . . . . . 7.1 Results for SPASS-IQ's pivoting strategies on the SMT-LIB benchmark division QF LRA . . . . . . . . . . . . . . . . . . 7.2 Results for SPASS-IQ's pivoting strategies on the SMT-LIB benchmark division QF LRA without switching to Bland's rule after finitely many pivots . . . . . . . . . . . . . . . . . . 7.3 Results for SPASS-IQ's pivoting strategies on the SMT-LIB benchmark division QF LIA . . . . . . . . . . . . . . . . . . . 7.4 Results for SPASS-IQ's pivoting strategies on the SMT-LIB benchmark division QF LIA without switching to Bland's rule after finitely many pivots . . . . . . . . . . . . . . . . . . 7.5 Greedy vs. lazy conflict detection on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.6 Greedy vs. lazy conflict detection on the QF LIA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.7 Multiple conflicts vs. single conflict on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 7.8 Multiple conflicts vs. single conflict on the QF LIA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.9 Another simplex backtrack function . . . . . . . . . . . . . . 7.10 Backtracking via recalculation (HardBacktrack()) vs. backtracking via backup (Backtrack()) on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.11 Backtracking via recalculation (HardBacktrack()) vs. backtracking via backup (Backtrack()) on the QF LIA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.12 Backtracking via recalculation (HardBacktrack()) vs. backtracking via backup (Backtrack()) on the QF LIA convert benchmark family . . . . . . . . . . . . . . . . . . . . . . . . . 7.13 FLINT vs. GMP on the QF LRA SMT-LIB benchmarks . . . 7.14 FLINT vs. GMP on the QF LIA SMT-LIB benchmarks . . . XV 7.15 Data structures for variables and bounds . . . . . . . . . . . . 7.16 Data structures for simplex tableaux and their entries . . . . 7.17 An efficient row summation function . . . . . . . . . . . . . . 7.18 Data structures for incrementally updated variable information225 7.19 A function that efficiently determines whether a basic variable and its defining row describe a conflict . . . . . . . . . . . . . 7.20 Data structures for the branching tree . . . . . . . . . . . . . 7.21 A control flow graph of the combination of techniques focused on the preprocessing procedures. The calls to the main loop (Figure 7.22) are labeled as branch-and-bound. . . . . . . . . 7.22 A control flow graph that describes the main loop of our combination of techniques, i.e., the procedures we apply in SPASS-IQ during every iteration of branch-and-bound. . . . . 7.23 With(out) branch-and-bound extensions on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 7.24 With(out) branch-and-bound (BnB) on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 7.25 With(out) simple rounding on the QF LIA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.26 Results for different bound propagation settings on the SMT-LIB benchmark division QF LIA . . . . . . . . . . . . . . . . 7.27 With(out) unit cube test on the QF LIA SMT-LIB benchmarks244 7.28 With(out) bounding transformations on the QF LIA SMT-.2 A control flow graph that describes SPASS-SATT's solver interactions during the propagation phase. . . . . . . . . . . . . 8.3 A control flow graph that describes SPASS-SATT's solver interactions during the theory verification phase. . . . . . . . . 8.4 A control flow graph that describes SPASS-SATT's solver interactions during the decision phase. . . . . . . . . . . . . . . 8.5 A control flow graph that describes SPASS-SATT's solver interactions during the conflict phase. . . . . . . . . . . . . . . 8.6 With(out) Decision Recommendation on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 8.7 With(out) Decision Recommendation on the QF LIA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 8.8 With(out) Unate Propagation on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.9 With(out) Unate Propagation on the QF LIA SMT-LIB ben-Results for different bound refinement settings on the SMT-LIB benchmark division QF LRA . . . . . . . . . . . . . . . . 8.11 Results for different bound refinement settings on the SMT-LIB benchmark division QF LIA . . . . . . . . . . . . . . . . 8.12 With(out) preprocessing techniques on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 8.13 With(out) preprocessing techniques on the QF LIA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.14 With(out) extensive preprocessing on the QF LIA nec smt benchmark family . . . . . . . . . . . . . . . . . . . . . . . . . 8.15 With(out) constant if-then-else (CITE) simplifications on the QF LIA nec smt benchmark family . . . . . . . . . . . . . . . 8.16 With(out) if-then-else (ITE) compression on the QF LIA nec smt benchmark family . . . . . . . . . . . . . . . . . . . . . . . . . 8.17 With(out) extensive preprocessing on the QF LIA rings benchmark family . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.18 With(out) shared monomial lifting on the QF LIA rings benchmark family . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.19 With(out) constant if-then-else (CITE) bounding on the QF LIA rings benchmark family . . . . . . . . . . . . . . . . . . . . . 8.20 With(out) if-then-else (ITE) reconstruction on the QF LIA rings preprocessed benchmark family . . . . . . . . . . . . 8.21 With(out) pseudo-boolean preprocessing on the QF LIA pb2010 benchmark family . . . . . . . . . . . . . . . . . . . . . . . . . 8.22 With(out) defined variable simplifications on the QF LRA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . 8.23 With(out) defined variable simplifications on the QF LIA SMT-LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 8.24 With(out) small CNF on the QF LRA SMT-LIB benchmarks 8.25 With(out) small CNF on the QF LIA SMT-LIB benchmarks 8.26 With(out) small CNF on the QF LIA convert benchmark family .

LIB benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 A control flow graph that describes the four phases of CDCL(LA) in SPASS-SATT. . . . . . . . . . . . . . . . . . . . . . . . . . 8chmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . XVI 8.10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 SMT-COMP 2018 results for QF LRA (main track, sequential, benchmarks: 1649, time limit: 1200s) taken from http:

Algorithm 6 :

 6 AssertLower(x i , l i ) Input : A variable and a new lower bound value to assert. Output : Returns false if the new lower bound for x i violates x i 's current upper bound. Returns true if the current assignment β satisfies the new bound. Otherwise, returns unknown. Effect : Adds the current lower bound to the old bound sequence and sets L to

  return true; 7 if x i ∈ N then update(x i , l i ); Returns false if the new upper bound for x i violates x i 's current lower bound. Returns true if the current assignment β satisfies the new bound. Otherwise, returns unknown. Effect : Adds the current upper bound to the old bound sequence and sets U to

	8 return unknown
	Algorithm 7: AssertUpper(x i , u i )
	Input : A variable and a new upper bound value to assert.
	Output :

T x ≤ -l. Otherwise, it is called unbounded . Definition 2.8.2 (Bounded System 19 ). A constraint system C is bounded if all directions h ∈ Q n \ {0 n } are bounded (see Figure 2.7 for an example). Otherwise, it is called unbounded (see Figures 2.6 and 2.8 for examples).

	x 2	x 2	x 2
	h
	x 1	x 1	x 1
	Figure 2.6: A partially	Figure 2.7: A boun-	Figure 2.8: An absolu-
	bounded system; the di-	ded system; all directi-	tely unbounded system;
	rections h = (-1, 1) T	ons are bounded	all directions are un-
	bounded directions in and -h are the only		bounded
	the example
	Unbounded problems are typically those problems, where decision proce-
	dures for linear integer/mixed arithmetic have problems with termination or
	at least with efficient termination. For instance, branch-and-bound without
	extensions diverges on most unbounded problems:

  2); • the Propagate & Propagate-Div rules are limited to δ-relevant bounds; • the Forget rule is never used to eliminate resolvents introduced by Resolve-Cooper; • it only applies the Conflict and Conflict-Div rules if Resolve-Cooper is not applicable. However, we are now going to discuss four examples where CutSat diverges despite strong conflict resolution. The first one shows that CutSat can apply Conflict and Conflict-Div infinitely often to constraints containing unguarded variables.

	Example 3.3.5. Let
	C

  CutSat++ cannot apply Resolve-Weak-Cooper again to (x 2 , {J 1 , J 3 }) because y 1 , which is a variable smaller than x 2 , is not yet fixed. Therefore, CutSat++ continues to propagate bounds for y 1 derived from the constraints K 1 and K 2 . Moreover, CutSat++ has to first propagate the newly introduced constraints K 1 and K 2 because CutSat++ follows a reasonable strategy and K 1 and K 2 are of the form ±y i + b i ≤ 0. Note that CutSat++ cannot propagate bounds for y 1 from K 3 and K 4 because x 1 is the top variable in those constraints. Instead, CutSat++ fixes y 1 with a decided bound. CutSat++ is still unable to apply Resolve-Weak-Cooper to (x 2 , {J 1 , J 3 }) because K 4 is a conflict with top(K 4 ) = x 1 ≺ x 2 .However, CutSat++ can and will apply Resolve-Weak-Cooper to the conflicting core (x 1 , {K 4 , I 7 }). The resulting resolvent is (R y 2 , R 2 ) for which CutSat++ introduces the fresh variable y 2 and updates the variable order tox 3 ≺ x 4 ≺ y 2 ≺ y 1 ≺ x 5 ≺ x 1 ≺ x 2 .Resolve-Weak-Cooper also truncates the bound sequence to the largest prefix that contains only decided bounds for variables x j with x j ≺ y 2 . Again, CutSat++ has to first propagate all constraints containing only one variable, so K 1 , K 2 , K 5 , and K 6 . Then, CutSat++ fixes y 2 with the decided bound y 2 ≥ 2. Since y 2 is now fixed, CutSat++ can use the divisibility constraint K 7 to refine the lower and upper bound of y 1 to 1. Similarly, fixing y 1 allows Cut-Sat++ to propagate a new lower bound for x 5 from K 8 . CutSat++ cannot propagate any further bounds for x 5 , so it fixes x 5 with a decided bound. Fixing x 5 allows CutSat++ to propagate an upper and a lower bound for x 1 from I 7 and K 4 , respectively. However, in the resulting state, (x, {K 3 , K 4 , I 7 }) is a conflicting core, which CutSat++ will resolve with Resolve-Weak-Cooper. The resulting resolvent is (R y 3 , R 3

C 3 Figure 3.16: The fourth part of a CutSat++ run for Example 3.5.11 sulting search state,

  Since Ax ≤ b contains cubes C e (z) for all e ∈ R, it holds for all e ∈ Q δ that width(h, Q

	.8)
	Therefore, width(h, C e (z)) = e • h 1 , which is independent of z. After inserting (4.7) and (4.8) into (4.6), we get:
	lim

e→∞ width(h, C e (.)) = lim e→∞ e • h 1 → ∞ . δ (Ax ≤ b)) ≥ width(h, C e (.)) ,

and, thus, width(h, Q δ

  .26 of Chapter 7.2.5, we examined the impact

	All instances of SMT-LIB division QF LRA
	2,400				0(924/669)
	2,000				1(935/673)
					10(935/673)
	1,500				100(default)(934/673) 1000(934/673)
	1,000				∞(935/673)
	500
	0	0	500	1,000	1,400 1,649
	horizontal axis: # of solved instances; vertical axis: time (seconds)
	For each setting the 249 hardest instances of QF LRA
	2,400
	2,000
	1,500
	1,000
	500
	1,400 0			1,649
	horizontal axis: # of solved instances; vertical axis: time (seconds)

  1 . . . t n ) with arbitrarily many operands, which is semantically interpreted as t 1 ∧ . . . ∧ t n ; • a disjunction operator (or t 1 . . . t n ) with arbitrarily many operands, which is semantically interpreted as t 1 ∨ . . . ∨ t n ; • a boolean negation operator (not t 1 ), which is semantically interpreted as ¬t 1 ; • an implication operator (=> t 1 t 2 ), which is semantically interpreted as t 1 → t 2 ; • an equivalence operator (= t 1 . . . t n ), which is semantically interpreted as (t 1 ≡ t 2 ) ∧ . . . ∧ (t n-1 ≡ t n ). The standard arithmetic operators in the ground linear arithmetic subset are: • a summation operator (+ t 1 . . . t n ) with arbitrarily many operands, which is semantically interpreted as t 1 + . . . + t n ; • a multiplication operator ( * t 1 . . . t n ) with arbitrarily many operands, which is semantically interpreted as t 1 • . . . • t n and which has at most one subterm that is a variable;

  and t 1 . . . t n )) → (or (not t 1 ) . . . (not t n )) (not (or t 1 . . . t n ))

	not (not t)) (not false) (not true) (not (	→ t → true → false

  13 . . . t k true t 1 . . . t m ) → true (ite t 1 t 1 t 2 ) → (ite t 1 true t 2 ) (ite t 1 (not t 1 ) t 2 ) → (ite t 1 false t 2 ) (ite t 1 t 2 (not t 1 )) → (ite t 1 t 2 true) (ite t 1 t 2 t 1 ) → (ite t 1 t 2 false) (and t 1 . . . t k true t 1 . . . t m ) → (and t 1 . . . t k t 1 . . . t m ) (or t 1 . . . t k false t 1 . . . t m ) → (or t 1 . . . t k t 1 . . . t m ) Apply associativity: (and t 1 . . . t k (and t 1 . . . t m ) t1 . . . tn ) → (and t 1 . . . t k t 1 . . . t m t1 . . . tn ) (or t 1 . . . t k (or t 1 . . . t m ) t1 . . . tn ) → (or t 1 . . . t k t 1 . . . t m t1 . . . tn ) Replace operators:

	Reduce to truth value:
	(not false) (not true) (and ) (and t 1 . . . t k false t 1 . . . t m ) (or ) (or t 1 Reduce to subterm:	→ true → false → true → false → false
	(not (not t)) (and t . . . t) (or t . . . t) (ite true t 1 t 2 ) (ite false t 1 t 2 ) (ite t 1 true false) (ite t 1 t 2 t 2 ) (ite (not t 1 ) t 2 t 3 ) Remove arguments:	→ t → t → t → t 1 → t 2 → t 1 → t 2 → (ite t 1 t 3 t 2 )

  and t i : Integer (+ t 1 . . . t n ) : Integer if t 1 : Integer, . . . , t n : Integer (ite t 1 t 2 t 3 ) : Integer if t 2 : Integer, t 3 : Integer

  15 dist d (x, y), 15 div-derive(J, x j , , M ), 72 diverging, 47-49, 55, 76, 83 divisibility conflicting core, 78, 95 divisibility constraint, 16, 18, 22-23, 63 divisibility core resolvent, 88 division operator, 263 div-solve(x j , {I 1 , I 2 }), 85 (double-)bounded part, 173

	CDCL, 253 CutSat, 66 CutSat++, 66 Farkas' Lemma, 29 for Explanations, 31 for Q δ , 29 see incremental efficiency, guarded conflict resolution, 70, 69-eager top-level propagated state, 117, 117-119 eager top-level propagating strategy, 72 guarded constraint, 51 incrementally connected, 39 incrementally updating information, leaf (branch-and-bound), 44 leaf (CITE), 273 Learn (CutSat), 73 LRA, 4, 16, 127, 144, abbr. for (theory of) linear rational arithmetic, Nelson-Oppen method, 159-161, 186 non-basic variable, 27 non-negative linear combination (in-pivot(y i , z j ), 35 pivotAndUpdate(y i , z j , v), 35 pivoting, 35, 199-203 rotate, 139 abbr. for satisfiability mo-superterm set, 266 transition rule, 55 rational, 16, 261 Q(C), 20 Round (tight), 71 dulo theories library, surface point, 131 transition system, 55, 57-123 basic, 27 Q δ (C), 25 QF LIA, rounding heuristic, 46, 232, 241 SMTInterpol, 138, 190, 191, 201 system of constraints, 16 tropical-matrix, 187, 240, 245, 256 branching, 45, 231-232
	decision phase (CDCL(LA)), 253 decision recommendation, 253, 255-257 decrementally connected, 39 defined variable, 278 δ (infinitesimal parameter), 24 δ-mixed solution, 25 δ-rational solution, 25 δ-rationals (Q δ ), 24 depth-first selection strategy, 230 dillig, 137, 139, 190, 240, 244 diophantine conflicting core, 79, 83, 95 diophantine core resolvent, 88 diophantine equation, see diophantine representa-tion, diophantine equation handler, 146 diophantine representation, 23, 72, 88 directed acyclic graph, 27, 160 disjunction operator, 262, 279 distance, 15 distance function, Double-Bounded Reduction, 177, 173-177 DPLL(T), greedy renaming CNF transformation, improves(J, x j , , M ), 68 98, 115-119 edge length (cube), 54 end state, 55, 65 entailment, 28, 28-31, 63, 144, 167 see also implied constraint, general, 28, 144, 167 integer, 28, 63 mixed, 28 rational, 28, 144, 167 Entry, 222 EqBasis(Ax ≤ b), 150 equality, 19, 143-164 equality basis, 146, 164, 165-175 equality operator, 263, 274 equisatisfiable, 21 integer, 21 mixed, 21 rational, 21 equivalence, 21, 63, 144 integer, 21, 63 mixed, 21 rational, 21, 144 equivalence operator, 262 equivalent, see equivalence, Euclidean algorithm, 182 eval(I, s), 19 evaluation, 19 explicitly bounded, 50 explicitly bounded problem, see explicitly bounded sys-tem, explicitly bounded system, 51, see also guarded system, explicitly entailed, 28, see also explicitly implied, explicitly implied, 28, largest cube test, 131, 131-133, 135, lower bound, 18, 19, 27, 65 n-dimensional zero vector, 15 pb2010, 283, 290 rings, 272, 283, 286, 289 dulo theories, top(I), 79, 94 variable abbr. for satisfiability mo-see also explicitly entailed, fast cube tests, 125-141, see also largest cube test, unit cube test, Fast Library for Number Theory, see FLINT, first-order theory, 2 fixed, 65 polynomial, 65 variable, 65, 231 FixEqs(y i ), 157 flat cube, 54, 134 FLINT, 218, 219 FlippedQFLIA, 190, 192 FlippedRandomUnbd, 190, 192 focused representation, 18, 63 Forget (CutSat), 73 Forget (CutSat++), 73 formula if-then-else, 263, 272 Fourier-Motzkin elimination, 60, 65, 128, 145 Gaussian elimination, 146, 172, 178 general entailment, 28, 144, 167 geometric object (axis-parallel hyper)cube, 53, 128-135 d-norm ball, 52 flat cube, 54, 134 GLPK, 140, 192 GMP, 218, 219 GNU Multiple Precision Library, see GMP, greater than operator, 263 greater than or equal operator, 263 greedy conflict detection, 209, 209-210 greedy pivoting strategy, 201, 203 guarded problem, see guarded system, guarded system, 51, see also explicitly bounded system, guarded variable, 51 Gurobi, 140, 192 HardBacktrack(d ), 213 hardware float, 217 hardware integer, 217 highlighted representation, 18, 145 highly incremental, see incremental efficiency, if-then-else arithmetic, 263, 272 formula, 263, 272 if-then-else elimination, 272 if-then-else operator, 263 if-then-else preprocessing (standard) elimination, 272 basic simplifications, 269 bounding, 276, 277, 286 constant simplifications, 273, 283 lifting shared monomials, 276, 286 reconstruction, 272, 289 if-then-else reconstruction, 272, 289 if-then-else simplifications, 269 implication operator, 262 implicitly bounded, 50 implicitly entailed, 28 implicitly implied, 28 implied constraint, 28, 28-31 see also entailment, equality, 143-164 implies, see implied constraint, 225 inequality system, 26, 26, also called polyhedron, pro-blem, (standard) constraint system, system of inequali-ties, Initialize(), 156 inprocessing procedures (BnB), 234, 236-237 input problem standard, 26 tableau representation, 27 (input) problem (SMT solver), 261 (input) problem (theory solver), 26 int, 219 Integer, 219 integer entailment, 28, 63 integer equisatisfiable, 21 integer equivalence, 21, 63 integer gap, 181 integer implied, see integer entailment, integer satisfiability, 20, 63 integer solution, 20 integer variable, 16, 261 interaction (CDCL(LA)), 249-260 interior point, 132 interval conflicting core, 78, 95 IsConflict(var i), 227 justification, 158 tight, 61, 70 L(x j ), 27 L(x j , M ), 65 LA, 2, abbr. for (theory of) linear arithmetic, Learn (CutSat++), 73 learning multiple conflicts, 209, 251 less than operator, 263 less than or equal operator, 263 let operator, 263 let variable, 263, 264 let-elimination alternative, 266, 266-267 standard, 265 LIA, 4, 16, 63, 127, abbr. for (theory of) linear integer arithmetic, lifting shared monomials, 276, 286 linear arithmetic, 2, abbr. LA, integer, 4, 16, 63, 127, abbr. LIA, mixed, 4, 16, 127, 167, abbr. LIRA, rational, 4, 16, 127, 144, abbr. LRA, linear combination inequalities, 29 vectors, 50, 163 linear cube transformation, 130 linear integer arithmetic, 4, 16, 63, 127, abbr. LIA, linear mixed arithmetic, 4, 16, 127, 167, abbr. LIRA, linear rational arithmetic, 4, 16, 127, 144, abbr. LRA, linear virtual substitution, 163 LIRA, 4, 16, 127, 167, abbr. for (theory of) linear mixed arithmetic, equalities), 29 non-strict inequality, 16, 22-26 pivoting cycle, 201 pivoting rule, abbr. for quantifier-free li-near integer arithmetic, solution, 19 system of equations, 26 two-layered strategy, 79 conflicting, 79, 95 run (transition system), 55 δ-mixed, 25 system of inequalities, 26, 26, type, 16 defined, 278 M(C), 20 M δ (C), 25 MathSAT, 11, 138, 190 maximum error rule, 202 maximum norm, 15 minimal conflict (explanation), 31, see also minimal set of unsa-tisfiable inequalities, minimal set of unsatisfiable inequali-ties, 30, see also minimal conflict (ex-planation), minimum error rule, 201, also called CVC4's rule, miplib, 256 miplib2003, 240, 241, 256 mixed column transformation matrix, 170 mixed entailment, 28 mixed equisatisfiable, 21 mixed equivalence, 21 mixed implied, see mixed entailment, mixed satisfiability, 20 mixed simple rounding, 16, 46, 134, 232 mixed solution, 20 Mixed-Echelon-Hermite normal form, 171 Mixed-Echelon-Hermite transforma-tion, 172, 178-183 multiple precision arithmetic libraries, 217 multiplication operator, 262 N , 27 n-dimensional all-one vector, 15 norm 1-norm, 15 d-norm, 15 maximum norm, 15 O, 40 omega test, 128, 168, 191 ω(x j ), 40 operator arithmetic division, 263 multiplication, 262 negation, 263 subtraction, 263 summation, 262 Boolean conjunction, 16, 262, 275, 277, 279 disjunction, 262, 279 equivalence, 262 implication, 262 negation, 262, 265 comparison equality, 263, 274 greater than, 263 greater than or equal, 263 less than, 263 less than or equal, 263 special if-then-else, 263 let, 263 partially unbounded problem, 52, 165-195 partially unbounded system, 52, 165-195 see pivoting strategy, pivoting strategy, 200, 200-203 greedy, 201, 203 terminating, 201 pivoting variable, 199 polyhedron, 22, 26, 52, 127, 144, also called inequality system, problem, (standard) con-straint system, system of in-equalities, positive occurrence, 265 potential conflicting core, 95 Preprocessing (SPASS-SATT), 261-294 preprocessing procedures (BnB), 234, 234-236 prime-cone, 137, 240, 244 Princess, 190, 191 problem (SMT solver), 261 problem (theory solver), 26, also called inequality system, polyhedron, (standard) con-straint system, system of in-equalities, Propagate (CutSat), 67 Propagate (CutSat++), 67 Propagate-Div (CutSat), 67 Propagate-Div (CutSat++), 67 propagated bound, 66 propagation phase (CDCL(LA)), 249, 252-254 propositional variable, 261, 264 pruned node, 44 pseudo-boolean variable, 278 returning multiple conflicts, 209, 251 abbr. for superposition mo-top variable, 79, 94 Var, 220 SMT, 247-294, pseudo-boolean problem, 278, 290 ded resolvent, SUP(T), 5, tightrhombus, 240 small CNF transformation, 280, 293 pseudo-boolean inequality, 279 see benchmarks, QF LIA, QF LRA, abbr. for quantifier-free li-near rational arithmetic, see benchmarks, QF LRA, Qvars(C), 17 RandomUnbd, 187, 190 Rational, 219 rational entailment, 28, 144, 167 rational equivalence, 21, 144 rational gap, 180 rational relaxation, 20 rational satisfiability, 20, 144 rational solution, 20 rational variable, 16, 261 rationally equisatisfiable, 21 rationally implied, see rational entailment, reasonable strategy, 73 relaxation, 20 RemoveEntry(entry k), 224 Resolve (CutSat), 69 Resolve (CutSat++), 69 resolve(γ, I), 70 Resolve-Cooper (CutSat), 78 Resolve-Implied (tight), 71 Resolve-Weak-Cooper (CutSat++), 94 resolvent, 86, 90, see also diophantine core re-solvent, divisibility core re-solvent, interval core resol-vent, strong resolvent, unguar-S d δ-rational, 25 also called inequality system, integer, 16 fixed, 65, 231 r (z), integer, 20 polyhedron, problem, (stan-rational, 16 guarded, 51 see also sphere, mixed, 20 dard) constraint system, let, 263, 264 Sat (CutSat), 74 Sat (CutSat++), 74 satisfiability, 19, 20, 63, 127, 144, rational, 20 solution set (for a specific variable), 89, 89-91 Tableau, 222 tableau (simplex), 27, 221-224 non-basic, 27 U(x j ), 27 pivoting, 199 U(x j , M ), 65 unate propagation, 252, 252 propositional, 261, 264 167, Solve-Div (CutSat), 78 tableau representation, 27 unbounded, 47-52 pseudo-boolean, 278 opposite: unsatisfiability, Solve-Div-Left (CutSat++), 94 term sharing, 265, 265-266 unbounded direction, 47, 162 slack integer, 20, 63 Solve-Div-Right (CutSat++), 94 terminating, 56, 111-116 unbounded part, 173 tableau representation, 27 mixed, 20 sound, 56, 122 terminating pivoting strategy, 201 unbounded problem, 47 top, 79, 94 rational, 20, 144 SPASS-IQ, 137-141, 187-194, 197-theory, 2 absolutely, 51, 135-141 unguarded, 51 satisfiability modulo theories, 247-294 247 theory of linear arithmetic, 2, partially, 52, 165-195 violated, 36, 199, 208 abbr. SMT, SPASS-SATT, 247-296 abbr. LA, unbounded system, 47 variable bound, 18, 42, 49, 65, 233 satisfiability modulo theories library, split system, 173 integer, 4, 16, 63, 127, absolutely, 51, 135-141 VarInfo, 225 abbr. SMT-LIB, standard CNF transformation, 279 abbr. LIA, partially, 52, 165-195 VarInfoIndices, 225 SCIP, 140, 192 standard conflict resolution, mixed, 4, 16, 127, 167, unguarded conflict, 79 vars(C), 17 search state, 66 see guarded conflict resolu-abbr. LIRA, unguarded conflict resolution, 93, 93-vector representation, 17, 127, 145, selected node, 45, 230-231 tion, rational, 4, 16, 127, 144, 94, 99 167 shared term, 265, 265-266 σ D,c standard if-then-else elimination, 272 abbr. LRA, unguarded constraint, 51 vertex, 129, 131, 136 y,z , 22 standard input problem, 26 theory of linear integer arithmetic, 4, unguarded problem, violated variable, 36, 199, 208 simple rounding, 15, 47, 131, 232, standard let-elimination, 265 16, 63, 127, see unguarded system, violated variable heap, 208, 208 241 standard representation, 17, 63, 127, abbr. LIA, unguarded system, 51 virtual substitution, 163 simplex algorithm, 34, 34-42, 199-226 simplex tableau, 27, 221-224 Skip-Decision (CutSat), 69 145, 167 start state, 55, 66 strategy (transition system), 56 strict inequality, 16, 22, 24-25 theory of linear mixed arithmetic, 4, 16, 127, 167, abbr. LIRA, theory of linear rational arithmetic, unguarded variable, 51 w-cooper(x j , C ), 95 unit cube test, 133, 133-135, 137-weak Cooper elimination, 86, 83-95, 139, 141, 177, 187, 232-233, 97 243-245 Skip-Decision (CutSat++), 69 slack variable strictly-two-layered strategy, 99 strong conflict resolution, 77, 77-83 4, 16, 127, 144, abbr. LRA, unit propagation, 249 Yices, 138, 190, 191 Unsat (CutSat), 74 tableau representation, 27 Slack-Intro (CutSat), 76 Slack-Intro (CutSat++), 76 SlackedQFLIA, 187, 190 slacks, 137, 187, 190, 191, 240, 244, summation operator, 262 tightly bounded, 154 upper bound, 18, 19, 27, 65 245 strong resolvent, 78 stuck state, 56, 82, 83, 121 stuck variable, 75 substitution, 22 subtraction operator, 263 theory solver, 5, 197-247 theory verification phase Unsat (CutSat++), 74 Unsat-Div (CutSat), 74 Z(C), 20 Z3, 11, 138, 190 (CDCL(LA)), 250, 250 Unsat-Div (CutSat++), 74 zero vector, 15 tight justification, 61, 70 update(z j , v), 35 tight(J, x j , M ), 70 unsatisfiability, 20 Zvars(C), 17
	see CDCL(T), DRational, 219 280, 293 147 nec smt, 256, 283 Q δ , 24, dulo theories, transition, 55 guarded conflict, 79 latendresse, 282, 291 negative occurrence, 265 see δ-rationals (Q δ ), SMT-LIB, transition relation, 55	incremental efficiency, 39 lower triangular matrix, 169 piv(A, j), 15 rings preprocessed, 272, 283, 289 superposition modulo theories, 5, arithmetic, 16, 264 F e (z), 54, see also flat cube, incrementality, with gaps, 169 pivot of a column, 15 root node, 44 abbr. SUP(T), integer, 16, 261

Any ground linear arithmetic formula can be transformed into a CNF problem with the help of a CNF transformation (see alsoChapter 8).

Unbounded directions are no obstacle for decision procedures over linear rational arithmetic because their termination strategies typically do not rely on enumerating all potential rational solutions.

40 minutes is also the time limit used by the SMT-COMP 2019 (the 14th International Satisfiability Modulo Theories Competition)

The exceptions are: we avoid confusion between row vectors and column vectors by simply writing row vectors as transposed column vectors; and we write 0 n for the ndimensional origin instead of 0 to highlight the dimension of the vector.

The semantic of a divisibility constrain di | ai1x1 + . . . + ainxn -bi can be summarized as ai1x1 + . . . + ainxn -bi is divisible by di.

We call the version of C where all variables are relaxed to rational variables the (rational) relaxation of C.

This means that the variables in our constraint systems are essentially existentially quantified for the standard goal of linear arithmetic.

Note that Lemma 2.1.1 still works for Q δ with the above definitions for . and dist d (., .). However, the range of dist d (., .) is only Q δ for d = 1 and d = ∞ and R δ = R × R otherwise.

Polyhedron is another alternative name for an inequality system. We use it mainly when we are looking at a system from a geometric perspective.

Note that the general definition for implied constraints is related to the rational solutions of the problem and not the mixed solutions. This is standard in the literature[START_REF] Schrijver | Theory of linear and integer programming[END_REF]. Later in this section, we will also define two other types of implied constraints that are related to mixed and integer solutions.

This lemma is also called the "affine" form of Farkas' Lemma[START_REF] Schrijver | Theory of linear and integer programming[END_REF].

In Chapter 7, we describe the implementation of our techniques in our theory solver SPASS-IQ. In Chapter 8, we describe the implementation of our own CDCL(LA) solver SPASS-SATT based on SPASS-IQ.

A theory solver is a decision procedure for the conjunctive fragment of the theory; so systems of linear inequalities in the case of linear arithmetic.

CDCL(LA) = CDCL(T) where the theory T is set to linear arithmetic (LA)

The first version of the dual simplex algorithm is much older[START_REF] Dantzig | A history of scientific computing[END_REF].

It is assumed that all versions of the simplex algorithm have an exponential worst-case runtime complexity because the simplex algorithm is NP-mighty[START_REF] Disser | The simplex algorithm is NPmighty[END_REF].

For more details where these assertions come from see Section 2.6.

A bound xj ≤ uj (xj ≥ lj) is subsumed if (xj ≤ u j ) ∈ C with u j ≤ uj ((xj ≥ l j ) ∈ C with l j ≥ lj)

The first condition is trivially fulfilled for every inequality if we employ the transformations described in Section 2.2.1

Note that the definitions for (un)bounded directions/problems are related to the rational solutions (i.e. rational entailment) of the problem and not the mixed solutions (i.e. mixed entailment).

[START_REF] Bézout | Théorie générale des équations algébriques[END_REF] In the literature, bounded systems are systems whose rational solutions fit into a finite hypercube/hyperball[START_REF] Schrijver | Theory of linear and integer programming[END_REF]. Our definition is equivalent but it is formulated based on (un)bounded directions. In our opinion, this alternative formulation is more useful for the structural analysis of (un)bounded systems.

Similarly, we call a constraint guarded if it contains only explicitly bounded variables. Otherwise, we call it an unguarded constraint.

The only exception are 1-norm balls.

The actual computational cost of a transition plays no role for a transition system.

For linear arithmetic problems over the rationals, such as the rational relaxation of an integer problem, the SMT and theorem proving communities already presented efficient approaches[START_REF] Dragan | Bound propagation for arithmetic reasoning in Vampire[END_REF][START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF].

See Chapter 2.10 for the basic definitions necessary to understand transition systems.

Note that the name CutSatg does not appear in[START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF]. We chose it to separate the two parts of the CutSat calculus more clearly.

Note that both CutSat and CutSat++ rely on this calculus as their foundation. Their only distinction is that CutSat uses strong conflict resolution and CutSat++ uses unguarded conflict resolution.

See Subsection 3.2.3 or the introduction of this chapter for the precise definition.

An expression/constraint/polynomial is fixed if all contained variables are fixed.

We postpone the definition of the function tight(J, xj, M ) (Fig.3.3) to Subsection 3.2.3 because justifications are first used explicitly in that part of this section.

Notice that z is a variable not occurring in the problem and only introduced for the calculation of div-derive(D, xj, , M ). Before the calculation of div-derive(D, xj, , M ) ends, z is eliminated. This means that div-derive never introduces a new variable to the problem although it introduces z for the intermediate calculations.

The restrictions to maximal variables in the definition of conflicting cores and to the Solve-Div rule were both confirmed as missing but necessary in a private communication with Jovanović.

Or a potential interval conflicting core if the divisibility constraint d | cjxj + sj is not part of the contradiction.

While termination requires that the order is fixed from the beginning for all unguarded variables, the ordering among the guarded variables can be dynamically changed.

For the sake of brevity, we simplified some of the constraints generated in this example. For instance, K3 is the simplified version of 4 | 2+3x1+3x3+4x4+y1. These simplifications have no influence on the overall behavior of CutSat++.

We call this operation mixed simple rounding and denote it by the function z n

as mentioned in Chapter 2.1.2.

http://www.spass-prover.org/spass-iq

If we combine the equality basis with a diophantine equation handler[START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF], then we even receive a substitution σ that eliminates the equalities in such a way that we can reconstruct an integer solution from them. The result is a new system of inequalities that implies no equalities and has an integer solution if and only if Ax ≤ b has one.

Actually, the literals we assert are full inequalities a T i x ≤ bi. Due to slacking, the left side of those constraints is abstracted to a slack variable s such that s = a T i x. The definition of the slack variable s = a T i x is directly stored in the simplex tableau and only a bound s ≤ bi remains as the literal for the SMT solver (see Chapter 2.4.2 for more details).

See Chapter 2.8 for an introduction into the terminology and formal basis of (un)bounded directions.

There actually exist various versions of linear virtual substitution, each based on a different case distinction over the values the variable y can assume. For the presented application of the equality basis, the actual version of linear virtual substitution is irrelevant.

See Lemma

2.5.4 for the definition of a certificate of unsatisfiability

In our implementation, we do actually use less efficient, Gaussian-elimination-based transformations instead of Bareiss algorithm and the algorithm of Kannan and Bachem. The reason is that these transformations are incrementally efficient (see Section 6.4). Our experiments show that the transformation cost still remains negligible in practice.

Initially, our system of inequalities Ax ≤ b is the empty set, its MEHNF Hy ≤ u is also the empty set, and our transformation matrix V is just the n × n identity matrix.

Without these restrictions, our transformations would not be mixed equisatisfiable!

When the size of coefficients in V gets too large, it can make sense to recompute H and V to get a smaller transformation matrix.

As mentioned in Chapter 2.7, we recommend to use the version of the dual simplex algorithm presented by Dutertre and de Moura (see Chapter 2.7.1 and[START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]) as the basis for the underlying branch-and-bound solver (see Chapter 2.7.3). We do so because this version is highly incrementally efficient.

Only incomplete models do not have to be checked. Complete models still need to be checked for complete theory satisfiability!

SPASS-IQ and all benchmark files, benchmark scripts, and benchmark results are available on http://www.spass-prover.org/spass-iq

Our benchmarks for quantifier-free linear integer arithmetic (SlackedQFLIA and RandomUnbd) were recently added to the SMT-LIB as part of the 20180326-Bromberger benchmark family.

Depending on the types of the input variables, we split the theory of linear arithmetic into three highly related, but ultimately different subtheories: the theory of linear rational arithmetic, which contains all input problems with only rational variables; the theory of linear integer arithmetic, which contains all input problems with only integer variables; and the theory of linear mixed arithmetic, which contains all input problems with both types of variables. SPASS-IQ is divided analogously into two main components: an implementation of the simplex algorithm for handling linear rational arithmetic (see Section 7.1) and an implementation of the branch-and-bound algorithm for handling linear mixed/integer arithmetic (see Section 7.2).The division between the two components is in all truth not that strict. The main focus of the simplex algorithm may be handling linear rational arithmetic, but we also use it for the overall representation and storage of variables, bounds, and constraints. It is, therefore, the foundation of SPASS-IQ.

In our implementation, this finite number is equal to the number of non-basic variables.

This pivoting rule is only used in CVC4 for the Dutertre and de Moura version of the simplex algorithm. CVC4 typically uses a different version of the simplex algorithm called sum of infeasibility simplex[START_REF] King | Simplex with sum of infeasibilities for SMT[END_REF] and we cannot reproduce its pivoting strategy in the Dutertre and de Moura version of the simplex algorithm.

This pivoting rule is also implemented in CVC4[START_REF] Barrett | CVC4. In CAV[END_REF] but not activated by default.

Note that this is a property that impacts the efficiency of SPASS-IQ as a theory solver inside a CDCL(LA) extension (e.g., SPASS-SATT) but not its efficiency as a standalone solver.

Most other SMT theory solvers also represent integers and rationals internally with the help of arbitrary precision arithmetic libraries[START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].

Only representing the non-zero coefficient is advantageous because our coefficient matrices are typically sparse, i.e., many coefficients are zero.

This scheme is also used in other SMT theory solvers, e.g., in veriT[START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF].

Our actual implementation also always minimized the common row denominators. We skip this step in the pseudocode of AddMultipleOfRowToRow() to keep the pseudocode concise and readable.

The third case might also look like it contains non-determinism, but it does not because we explore the nodes depth-first and every node has at most two children.

We compute and store this preference before we start branch-and-bound as the variables preferred branching direction.

Bound Refinement/Propagation is used in most SMT theory solvers[START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Cimatti | The MathSAT5 SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].

A bound xj ≤ uj (xj ≥ lj) is subsumed if (xj ≤ u j ) ∈ C with u j ≤ uj ((xj ≥ l j ) ∈ C with l j ≥ lj)

We skip this step if all original integer variables are explicitly bounded.

Note that simple rounding does not change the assignment if the rational solution is already a mixed solution.

This also includes the simplex computations for the unit cube test. We adjusted the bound values of the dynamic instance based on the static instance and let the simplex algorithm run on the dynamic instance.

Note that we cannot simply overwrite the static and the dynamic instance of the original constraints. We need the original static instance as a copy of all original constraints for the iterative extension of the Mixed-Echelon-Hermite transformation. We need the original dynamic instance to store and to incrementally extend the current bounded basis, which is also needed for the iterative extension of the Mixed-Echelon-Hermite transfor-

mation.[START_REF] Berman | The complexity of logical theories[END_REF] Note that the iterative extension of the Mixed-Echelon-Hermite transformation occurs only if we combine multiple theories, which we do not do in the current version of SPASS-SATT.

These statistics do not include instances that may require branch-and-bound but could not be solved in the given time limit.

These instances are 162 of the 806 20180326-Bromberger benchmarks which include 303 partially unbounded problems[START_REF] Bromberger | A reduction from unbounded linear mixed arithmetic problems into bounded problems[END_REF], 1 of the 100 arctic-matrix benchmarks which include at least 2 partially unbounded problems[START_REF] Codish | Exotic semi-ring constraints[END_REF], 3 of the 93 cut lemmas benchmarks which include 5 partially unbounded problems[START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF], 1 of the 233 slacks benchmarks which include 3 partially unbounded problems[START_REF] Jovanović | Cutting to the chase -solving linear integer arithmetic[END_REF], and 2 of the 107 tropical-matrix benchmarks which include at least 4 partially unbounded problems[START_REF] Codish | Exotic semi-ring constraints[END_REF] .

Other unique features of SPASS-SATT, viz., unit cube test and bounding transformations, are presented in Chapter 7, where we describe SPASS-IQ, the theory solver used inside SPASS-SATT.

For a general description of CDCL(T) see Chapter 2.6.

We gave a stand-alone and in-depth description of the theory solver SPASS-IQ in Chapter 7 and an in-depth description of the SAT solver used by SPASS-SATT can be found in[START_REF] Wagner | Design and implementation of a CDCL(LA) calculus[END_REF].

Can be done automatically and efficiently with the two-watched literals scheme[START_REF] Moskewicz | Chaff: Engineering an efficient SAT solver[END_REF]

Can also be done automatically and efficiently with the two-watched literals scheme[START_REF] Moskewicz | Chaff: Engineering an efficient SAT solver[END_REF]

This series of bound refinements only propagates bounds that correspond to already existing literals in the current set of clauses. Moreover, it propagates at most 100 bounds for each variable.

In SPASS-SATT we do not add the unate propagation clauses explicitly to the set of clauses. This is not necessary because the theory solver can efficiently recompute them on the fly when we need them. Therefore, storing them explicitly would only wast space[START_REF] Gent | Lazy explanations for constraint propagators[END_REF].

Can be done efficiently (i.e., constant time) with the watched literals scheme[START_REF] Moskewicz | Chaff: Engineering an efficient SAT solver[END_REF].

Note that we use two different ids if two variables are defined by two different let operations. This means our internal representation does not have to handle overwritten variables.

We also implement the other types of terms internally through a pair consisting of an operator and an argument array. To do so, we create operators for the variables, the truth values, and one operator each for rational and integer numbers. The variables, the truth values have an empty argument array. The number operators store the numerical value as their first argument.

Most SMT solvers use similar shared term representations[START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].

Most SMT solvers use similar simplification techniques[START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF].

These simplifications require only constant time because the arguments of our subterms are shared terms, i.e., two equivalent arguments point to the same subterm.

[START_REF] Beale | An alternative method for linear programming[END_REF] Many SMT solvers flatten their arithmetic terms in a similar fashion, e.g., Yices[START_REF] Dutertre | Yices 2.2[END_REF].

Linear in the number of monomials that are part of the operation.

All of our if-then-else preprocessing techniques are based on CVC4's if-then-else preprocessing techniques[START_REF] Barrett | CVC4. In CAV[END_REF].

We can do a similar elimination for if-then-else operations, where either the consequence or alternative is true. However, SPASS-SATT did not gain any advantage on the SMT-LIB benchmarks for also treating these cases.

This definition assumes for simplicity that the shared monomials q appear after the unshared monomials q and q. In reality this is not always the case and SPASS-SATT has to find and extract the shared monomials explicitly.

This is actually not a major problem because we can easily transform any model for the CNF formula into a model for the original formula by just ignoring the newly added propositional variables.

Other preprocessing techniques, e.g., constant if-then-else bounding, also have a negative effect on the nec smt benchmarks if we do not first apply the constant if-then-else simplifications and the boolean if-then-else compression.

Acknowledgments We also presented various applications for the equality basis y -Dz = c. (i) We can use the equality basis to eliminate all equalities from Ax ≤ b. It is, therefore, useful as a preprocessing step for our cube tests. (ii) We can use the equality basis to directly check whether an equality h T x = g is implied by Ax ≤ b. (iii) We can use the equality basis to efficiently compute all pairs of equivalent variables in Ax ≤ b (Section 5.4). These pairs are necessary for a backjump-free Nelson-Oppen style combination of theories. (iv) We can use the equality basis to efficiently compute a bounded basis for Ax ≤ b, i.e., a finite representation of all bounded directions in Ax ≤ b (Section 5.5). (v) We can use the equality basis for quantifier elimination (Section 5.6).

Conflict Extraction

CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0,

where the new order is x 3 ≺ x 4 ≺ y 3 ≺ y 2 ≺ y 1 ≺ x 5 ≺ x 1 ≺ x 2 , and

where S = M, C 3 is the state before the application of Propagate-Div, and K 12 := div-derive(K 12 , y 3 , ≤, M ) = 2y 1 + 4y 2 + y 3 -18 ≤ 0 =⇒ Decide CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8]], C 3 =⇒ Propagate CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, y 2 ≥ K 5 0, y 2 ≤ K 6 2, y 1 ≥ K 1 0, y 1 ≤ K 2 3, y 2 ≥ 2, y 1 ≤ K 7 1, y 1 ≥ K 7 1, x 5 ≥ K 8 1, y 3 ≥ K 9 0, y 3 ≤ K 10 11, y 3 ≤ K 12 8, y 3 ≥ 8, x 5 ≥ K [START_REF] Barrett | Satisfiability Modulo Theories[END_REF] 3]], C 3

where S = M, C 3 is the state before the application of Propagate, and K 13 := tight(K 13 , x 5 , M ) = -x 5 + y 3 + 6y 2 -17 ≤ 0 =⇒ Decide CS [[x 3 ≥ I 1 0, x 3 ≤ I 2 0, x 4 ≥ I 3 0, x 4 ≤ I 4 0, x 5 ≥ I 5 0, riables because we only need the bounds that can be part of an equality explanation, i.e., only bounds that hold tightly for all satisfiable assignments (see Lemma 5.2.3). For the same reason, Initialize() eliminates all originally strict bounds, i.e., bounds with a non-zero delta part.

Next, Initialize() tries to turn as many variables x i with L(x i ) = U(x i ) into non-basic variables. We do so because x i is guaranteed to stay a non-basic variable if L(x i ) = U(x i ) (see lines 6 & 12 of Check). Pivoting like this essentially eliminates the tightly bounded non-basic variable x i and replaces it with the constant value L(x i ). There only exists one case when Initialize() cannot turn the variable y i with L(y i ) = U(y i ) into a nonbasic variable. This case occurs whenever all non-basic variables z j with non-zero coefficient a ij also have tight bounds L(z j ) = U(z j ). In this case, the complete row y i = z j ∈N a ij z j simplifies to x i = L(y i ), so it never produces a conflict and we can also ignore this row.

As its last action, Initialize() turns the bounds of all variables x j with L(x j ) = U(x j ) into strict bounds. Since Initialize() transformed these bounds into strict bounds, the condition of the while loop in line 3 of FindTBnds() checks whether the system contains another tightly bounded variable (see also Lemma 5.2.2). If Check returns (false, y i ), then the row y i represents an equality explanation and all variables z j with a non-zero coefficient in the row hold tightly (see Lemma 5.2.3). FindTBnds() uses FixEqs(y i ) (Figure 5.4) to turn these tightly bounded variables x k into explicit equalities by setting L(x k ) = U(x k ). After FixEqs(y i ) is done, we go back to the beginning of the loop in FindTBnds() and do another call to Check.

Algorithm 13: Initialize()

Effect : Removes all bounds l k and u k that cannot produce equalities; turns as many basic variables x i with l i = u i into non-basic variables as is possible; the bounds for all variables x k are turned into strict bounds if l k < u k Output : A backup of the original bound functions 1 L := L ; // Backup of the original lower bound function 2 U := U ; // Backup of the original upper bound function

We receive the set of tightly propagating bounds that we found with the last call to Check() if we turn all bounds in C into non-strict bounds:

From the proof of Lemma 5.2.3, we see that the bound

and, therefore, the clause

justifies the asserted bound x k ≤ L (x k ). Together with the slack variable definitions stored in the simplex tableau, this clause is a tautology and the SAT solver can learn it without restrictions. Moreover, all literals in this clause except for x k ≤ L (x k ) are asserted as unsatisfiable in the current model of our SAT solver. Therefore, the SAT solver can assert the literal x k ≤ L (x k ) on its own through unit propagation.

Symmetrically, FixEqs(y i ) asserts for every bound (x k ≤ U (x k )) ∈ C the bound x k ≥ U (x k ) and the justification for this bound is the clause:

Note also that all justifications we defined are in some sense minimal: each of the above clauses is a tautology and, if we remove one literal from the clause, then it is no longer a tautology. This fact is another property that enhances any potential conflict analysis.

Application: The Nelson-Oppen Method

In this Section, we explain how the integration of our methods in the simplex algorithm can be used for the combination of theories with the Nelson-Oppen method. For the Nelson-Oppen style combination of theories inside an SMT solver [START_REF] Bruttomesso | Delayed theory combination vs. Nelson-Oppen for satisfiability modulo theories: a comparative analysis[END_REF], each theory solver has to return all valid equations between variables in its theory. Linear arithmetic theory solvers sometimes guess these equations based on one satisfying assignment. Then the equations are transferred according to the Nelson-Oppen method without verification. This leads to a backtrack of the combination procedure in case the guess was wrong and eventually led to a conflict. With the availability of an equality basis, the guesses can be verified directly and efficiently. Therefore, the method helps the theory solver in avoiding any conflicts due to wrong guesses together with the overhead of backtracking. This comes at the price of computing the equality basis, which should be negligible because the integration we propose is incremental and includes justified simplifications.

An Incremental Version of the Mixed-Echelon-Hermite Transformation

The Mixed-Echelon-Hermite normal form (MEHNF) can also be computed incrementally efficient. However, the polynomial time algorithms [START_REF] Bareiss | Sylvester's identity and multistep integer-preserving Gaussian elimination[END_REF][START_REF] Kannan | Polynomial algorithms for computing the smith and hermite normal forms of an integer matrix[END_REF] for computing the reduced echelon column form and the hermite normal form are typically less incrementally efficient than the ones based on Gaussian elimination [START_REF] Xin | On the worst-case complexity of integer Gaussian elimination[END_REF]. So to achieve incremental efficiency, we have to accept a worst-case exponential runtime. Fortunately, the Gaussian based transformations rarely seem to reach their exponential worst-case in practice. Our experiments support this assumption since the transformation cost is negligible (if not immeasurable) on all of the tested benchmarks (see Section 6.5).

Algorithm 16: PivR(k, H)

Input : A row dimension k and a matrix H ∈ Q k×n 1 return If all columns j ≤ n 1 in H are 0 k , then 0 is returned.

Otherwise, this function returns the largest j such that column j in H is not 0 k and 0 < j ≤ n 1 .

Algorithm 17: PivI(k, H)

Otherwise, this function returns the largest j such that column j in H is not 0 k and n 1 < j ≤ n. The incrementally efficient version of the Mixed-Echelon-Hermite transformation (see ExtendMEH() in Figure 6.4)) works as follows: let Hy ≤ u be a system of inequalities in MEHNF and let V be an n × n mixed column transformation matrix such that AV y ≤ b is equivalent to Hy ≤ u. Then (H y ≤ u , V ) := ExtendMEH(Hy ≤ u, V, a T m+1 x ≤ b m+1 ) returns a system of inequalities in MEHNF H y ≤ u and a mixed column transformation matrix V such that A V y ≤ b is equivalent to H y ≤ u .

As already mentioned in Section 6.2, it is not possible to transform every matrix A ∈ Q m×n into Mixed-Echelon-Hermite normal form. We have to restrict ourselves to matrices, where the upper left r × n 1 submatrix has the same rank r as the complete left m × n 1 submatrix. However, this condition is very easy to fulfill because we are looking at systems of inequalities Ax ≤ b and not just matrices. This means we can simply swap the inequalities in Ax ≤ b to get the equivalent system Cx ≤ u, where C's upper left r × n 1 submatrix has the desired form. This swapping is also the reason why the input MEHNF Hy ≤ u and output MEHNF H y ≤ u of ExtendMEH() are only equivalent to AV y ≤ b and A V y ≤ b instead of the stronger conditions H = AV and H = A V we previously used for MEHNF's of matrices.

Otherwise, ExtendMEH() works the same as one step of the Gaussian based Mixed-Echelon-Hermite transformation for matrices: ExtendMEH() first applies the previous column transformations V to a T m+1 x ≤ b m+1 to get the inequality h T m+1 y ≤ b m+1 (line 1). Next, ExtendMEH() checks whether h T m+1 has any non-zero entries h m+1j in one of the zero columns j of H, i.e., in one Algorithm 18: ExtendR(Hy ≤ u, V, h T m+1 y ≤ b m+1 , j) Input : A system of inequalities Hy ≤ u, where H ∈ Q m×n is in Mixed-Echelon-Hermite normal form and u ∈ Q m δ ; a mixed column transformation matrix V ∈ Q n×n ; an inequality h T m+1 y ≤ b m+1 , where h m+1 ∈ Q n and b m+1 ∈ Q δ ; and a column index j ≤ n 1 such that h m+1j = 0 and h ij = 0 for all i ≤ m Effect : Extends Hy ≤ u by h T m+1 y ≤ b m+1 and transforms it into Mixed-Echelon-Hermite normal form H y ≤ u via mixed column transformation operations, which are stored in V . H has one more non-zero rational column than H and

of the columns j of H that consists only of zero entries. If h T m+1 does not have any such entries, then no column transformations are necessary and

In order to resolve this, we have to distinguish between two cases: Case 1: h T m+1 fills a rational gap of H, i.e., there exists a zero column 0 < j ≤ n 1 in H such that h k+1j = 0. In this case, we have to extend H from p -1 non-zero rational columns to p non-zero rational columns. We do so with the function ExtendR() (Figure 6.6). ExtendR() first inserts the inequality h T m+1 y ≤ b m+1 at an appropriate position p (line 3), to solve the rank requirements we discussed before. So in the new constraint system Algorithm 20: ReduceLeftI(H , V, p)

and a row and column index p. Effect : Negate all columns i ≥ p with h pi < 0. Extract the integer part s pi of each coefficient h pi . Store all non-zero integer parts s pi and their column index i in a set S. Output: (H ,V ,S) Input : The decision level to backtrack to. Effect : Reverts the bound value functions so they map to the bounds upto decision level d . Recomputes a satisfiable assignment for the decision level d . This is also the reason why most SMT theory solvers (e.g. CVC4 [START_REF] Barrett | CVC4. In CAV[END_REF] and veriT [START_REF] Bouton | veriT: An open, trustable and efficient SMTsolver[END_REF]) use a different backtracking technique (see HardBacktrack() in Figure 7.9). The HardBacktrack() function reverts the bound values the same way the original Backtrack() function does, but it does not revert the assignment to the backup solution assignment. Instead, it computes a potentially new solution with the Check() function.

HardBacktrack() itself is obviously slower than Backtrack(). However, the backtracked state returned by HardBacktrack() is sometimes a better continuation point for the simplex algorithm. The reason is that a good continuation point for the simplex algorithm depends not only on the current assignment, but also on the current tableau. Recomputing the solution with Check() keeps the assignment and the tableau in sync, which results in a good continuation point. Reverting to the backup solution does the opposite and sometimes disconnects the assignment and the tableau, which results in a bad continuation point. The coefficient matrix A of the tableau Az = y is implemented through several double-linked lists. To be more precise, each row and each column of the matrix A correspond to one double-linked list consisting of entries build from the data structure Entry (see Figure 7.16). Each such entry corresponds to one non-zero coefficient a ij of the coefficient matrix A. 6 Each entry is also part of two double-linked lists: one row list and one column list. To this end, the data structure Entry contains the two fields "int rowIdx, colIdx" that indicate the entries position in the coefficient matrix A and, therefore, also the row and column it belongs to.

Based on these indices, a row list contains all entries with the same row index rowIdx. The list itself is formed by the fields "Entry* rowPrev, rowNext" that point to the previous and next entry in the row. If rowPrev of a given entry points to NULL, then the entry is the first entry in the row. If rowNext of a given entry points to NULL, then the entry is the last entry in the row. Moreover, we can access a given row with index i from the tableau through its corresponding basic variable, i.e., rows [i].first is the first entry of the row list and rows [i].last the last entry.

The column lists are defined symmetrically, i.e., a column list contains all entries with the same column index colIdx. The list itself is formed by the fields "Entry* colPrev, colNext" that point to the previous and next entry in the column. If colPrev of a given entry points to NULL, then the entry is the first entry in the column. If colNext of a given entry points to NULL, then the entry is the last entry in the column. Moreover, we can access a given column with index j from the tableau through its corresponding nonbasic variable, i.e., columns [j].first is the first entry of the column list and columns [j].last the last entry.

We chose double-linked and not just single-linked lists because they make row summations more efficient. Or to be more precise, double-linked lists are more efficient at adding and removing single entries, especially, if the entries are linked in two lists. There are, however, three other steps that make row summations more efficient. As our first step, we reduce the number of mathematical operations needed for row summations by turning our rational

Storing and Incrementally Updating Information

The simplex algorithm regularly checks whether our tableau, bounds, and variables fulfill certain properties. For instance, whether and which bound of a basic variable is violated and whether this violation is already a conflict. All of these properties can be computed on a need to know basis. However, we typically check them more often than they actually change. This is bad because it can be quite expensive if we compute some of these properties regularly (e.g., every time linear in the number of non-zero row entries).

For this reason, we decided on a runtime efficient alternative. Instead of recomputing too expensive to compute information, we store a redundant copy of the information and update it when the actual changes happen. (Similar information storage and update schemes can be found in other SMT theory solvers, e.g., in SMTInterpol [START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF].) The drawback of this method is that it requires more memory. However, memory typically is a more readily available resource.

All of the information that we store and update is associated with one variable. We, therefore, store it as part of the variables structure Var. However, we try to avoid any confusion between the actually necessary fields and the redundantly stored information by moving the redundantly stored information into its own data structure VarInfo (see Figure 7.18).

Let us now discuss the fields of VarInfo and, therefore, the redundantly stored information. The first field "int boolInfo" of VarInfo is just a single integer. However, each of its bits corresponds to one boolean property. The stored boolean properties and their bit positions are stored in the enumeration VarInfoIndices: hasLower/hasUpper indicates whether the variable

Branch-and-Bound Implementation

SPASS-IQ's second set of decision procedures revolves around an implementation of the branch-and-bound algorithm, which we already explained in Chapter 2.7. We need this implementation of branch-and-bound because our simplex implementation alone can only handle linear rational arithmetic. If we, however, extend the simplex implementation with a branch-and-bound implementation, then we also get a decision procedure for the theories of linear integer and linear mixed arithmetic.

Most SMT solvers (e.g., [START_REF] Barrett | CVC4. In CAV[END_REF][START_REF] Christ | SMTInterpol: An interpolating SMT solver[END_REF][START_REF] Mendonça De Moura | Z3: an efficient SMT solver[END_REF][START_REF] Dutertre | Yices 2.2[END_REF]) implement branch-and-bound through a technique called splitting-on-demand [START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF], which delegates some of the branch-and-bound reasoning to a SAT solver. In order to keep more control over the branch-and-bound reasoning, we decided against splittingon-demand and implemented branch-and-bound without the help of a SAT solver (see Subsection 7.2.1). This also made it easier to complement branch-and-bound with other decision procedures: a simple rounding test (Chapter 2.7.4), a unit cube test (Chapter 4), a bound propagation procedure (Chapters 2.7.2 and 8.1.2), and our two transformations that reduce any problem into a bounded problem (Chapter 6). We have already discussed these extensions as standalone procedures in previous chapters and in these chapters we have also presented efficient ways to implement them. But we have not yet explained how we efficiently combine them. We remedy this in Subsection 7.2.4. Moreover, we discuss in Subsection 7.2.2 the selection strategies that we use in our branch-and-bound implementation. To be more precise, we explain the selection strategies that SPASS-IQ uses for selecting branching variables and active nodes.

Branching Tree Implementation

Most SMT solvers implement branch-and-bound through a technique called splitting-on-demand [START_REF] Barrett | Splitting on demand in SAT modulo theories[END_REF]. This means they typically implement their branching tree through branching clauses

x i ≤ c i ∨ x i ≥ c i and let a SAT solver guide the selection of nodes through the selection of literals.

Using a SAT solver for branch-and-bound enables better conflict learning in the CDCL(LA) framework (see Chapter 2.6). To be more precise, a SAT solver can globally learn local conflict explanations for pruned nodes. This results in two advantages: (i) the SAT solver dynamically combines the pruning conflicts into one conflict for the whole branching tree and (ii) the learned pruning conflicts enable the SAT solver to automatically repeat similar prunings in later branch-and-bound searches. The second advantage sounds useful in theory but similar prunings are actually rare in practice. One reason is that CDCL(LA) solvers rarely need more than a small number of subsequent branch-and-bound searches in practice.

Splitting-on-demand also results in one major disadvantage: the SMT solver has to add at runtime new literals to the SAT solver because the bounds in a branching clause are not necessarily existing literals. These new literals confuse the regular SAT solver search because (i) they are disconnected from the original clauses and (ii) their actual relevance is only short term (during the current branch-and-bound search) but they are added long term (for the remaining run). Moreover, they also confuse later branch-and-bound searches because the new literals force us to repeat old branches without considering the changed model. In our opinion, this disadvantage outweighs the advantages of splitting-on-demand and, therefore, our own branch-and-bound implementation is separate from the SAT solver. simplex algorithm as our proof. Next we try to round our rational solution to a mixed assignment. If this assignment is a solution, then we are also already done. We stop and return satisfiable with the rounded assignment as our proof. Otherwise, we try to apply the unit cube test as another shortcut to a potentially mixed solution. This test is important because it alone automatically solves all absolutely unbounded problems and some other problems too. We do, however, skip the unit cube test if one of the original integer variables has bounds U(x i ) -L(x i ) = 1 or if one of the slacked variables has bounds U(x i ) -L(x i ) = 0. In these cases, the unit cube test is guaranteed to fail and would just cost time. If some of the original integer variables are fixed, i.e., U(x i ) -L(x i ) = 0, then we treat them as if they were replaced by their constant values. This avoids another case where the unit cube test would otherwise automatically fail.

If the unit cube test fails, then we determine whether the problem is bounded and which directions are unbounded. To this end, we compute a bounded basis with the techniques proposed in Chapter 5.5. 13 then we use the transformations from Chapter 6 to reduce our problem to an equisatisfiable but bounded problem. The bounded problem (whether it is the original or the transformed problem) is then analyzed and we compute the preferred branching directions for all integer variables as explained in Subsection 7.2.2.

As our next step, we apply branch-and-bound extended by inprocessing procedures to the bounded problem (Figure 7.22). This is also the main loop of our combined procedure. We start the loop by adding our input problem as the root node of the branching tree and select it as our first selected node. The remaining branch-and-bound algorithm is structured as an iterative process for every selected node. In every iteration, we take our selected node and perform bound propagations on the subproblem C represented by the selected node (Chapter 2.7.2). We do, however, limit the number of propagated bounds so that we get at most a hundred propagated lower/upper bounds for each variable at the root node and at most ten propagated lower/upper bounds for each variable at all other nodes. Depending on the propagated bounds C , we now have to distinguish between two cases.

Chapter 8

Implementation of SPASS-SATT SPASS-SATT uses at its core a CDCL(LA) implementation that combines the CDCL (conflict-driven-clause-learning)-based SAT solver SPASS-SAT from the SPASS Workbench [START_REF]The SPASS Workbench[END_REF] with our LA theory solver SPASS-IQ. In this chapter, we explain the construction of SPASS-SATT in more detail. We start in Section 8.1 by outlining the interaction between the theory solver and the SAT solver in the CDCL(LA) implementation. This also includes extensions to the theory reasoning that enhance and guide the search of the SAT solver.

In Section 8.2, we explain the preprocessing techniques incorporated into SPASS-SATT. We start by giving a short introduction to the input language used by SPASS-SATT (SMT-LIB standard v2.0) [START_REF] Barrett | The SMT-LIB Standard: Version 2.0[END_REF] and specifically the term expressions we need to handle because of this language. The two most troublesome types of expressions introduced by the SMT-LIB language are let expressions and if-then-else expressions. They are so troublesome because CDCL(LA) cannot handle them directly and because their removal can cause an exponential blow-up in the formula size. The purpose of our first two sets of preprocessing techniques is, therefore, the intelligent removal of let expressions and if-then-else expressions.

The first preprocessing technique, which we present in Subsection 8.2.2, is just our internal term representation. This term representation is shared, i.e., we only need one copy for equivalent subterms; even if we have exponentially many occurrences of the same subterm. This allows us to remove let expressions and ignore the resulting exponential blow-up of the formula size because the number of different subterms does not increase.

Based on the shared term representation, there also exist some efficient simplifications that can reduce the overall formula size upto an exponential factor. The most important simplifications that reduce our formula size are presented in Subsection 8. nation with all of its literals still asserted. The explanation {L 1 , . . . , L n } is then removed from the backlog and turned into a conflict clause L1 ∨. . .∨ Ln . Moreover, CDCL(LA) then exits the theory verification phase and handles the conflict clause in the conflict phase. If the backlog contains explanations {L 1 , . . . , L n } and all of them are no longer conflicts, then the theory solver learns all of them, i.e., turns them into clauses L1 ∨ . . . ∨ Ln and adds them to the current clause set. Moreover, CDCL(LA) then exits the theory verification phase and tries to propagate the new clauses in the propagation phase.

Otherwise, the backlog contains no explanations and the theory solver verifies the theory satisfiability of the current model with SPASS-IQ. Note, however, that SPASS-SATT verifies only for full models whether they are linear mixed/integer satisfiable. Partial models are checked only for rational satisfiability. If SPASS-IQ returns unsatisfiable, then SPASS-IQ also found at least one conflict explanation. The theory solver then adds the conflict explanations to the conflict backlog and processes the conflict backlog as described before. If SPASS-IQ returns satisfiable instead, then the theory solver tries to refine the current model if it is partial. For this purpose, it first tries to compute a series of bound refinements (see Chapter 2.7.2). 6 If it succeeds, then bound refinement returns a propagation explanation {L 1 , . . . , L n } for each refined bound, which the theory solver then learns. Moreover, CDCL(LA) then exits the theory verification phase and tries to propagate the new clauses in the propagation phase.

If bound refinement fails, then the theory solver tries to greedily perform a series of unate propagations [START_REF] Dutertre | A fast lineararithmetic solver for DPLL(T)[END_REF]. A unate propagation is possible whenever the current model (i) contains a literal L corresponding to a bound x i ≤ u (x i ≥ l), (ii) the set of clauses contains another literal L corresponding to a bound x i ≤ u (x i ≥ l ) with u < u (l > l ), and (iii) neither L nor L is part of the current model. Clearly, the bound

) is also satisfied. The clause L ∨ L is, therefore, a tautology and we call it a unate propogation clause. If CDCL(LA) actually finds any unate propagations, then it learns the unate propogation clauses L ∨ L , exits the theory verification phase, enters the propagation phase, and lets the SAT solver propagate the literals L . 7 If CDCL(LA) finds no unate propagations, then it fails to refine the current model, exits the theory verification phase and enters the propagation phase.

Or second preprocessing step for CITEs uses the function csimp(o t 1 t 2 ), where t 1 and t 2 are CITEs and o is one of the operators <=, =, >=, to simplify all linear arithmetic atoms over at most two CITEs, i.e., the atoms defined by the following language:

catom :: The above constant if-then-else simplifications are a subset of the techniques presented by Kim et al. in [START_REF] Kim | Efficient term-ite conversion for satisfiability modulo theories[END_REF]. The same subset of constant if-then-else simplifications is also used by the SMT solver CVC4 [START_REF] Barrett | CVC4. In CAV[END_REF]. The above simplifications can only be performed efficiently if the intermediate results are computed once and not recomputed again. To this end, we cache all intermediate results in hash maps and check our cache before we recompute the results. For the constant if-then-else simplifications, we use two caches: (i) we cache the computed leaves of a CITE subterm t in a hash map pointing from shared terms to leaf sets; and (ii) we cache the recursive csimp(o t 1 t 2 ) calls in a hash map pointing from the triple (o, t 1 , t 2 ) to the return value of csimp(o t 1 t 2 ).

The rings Benchmark Family

The rings benchmark family was constructed in order to add some hard QF LIA problems to the SMT-LIB. To be more precise, the goal was to create integer problems that cannot be solved with only techniques for linear rational arithmetic [START_REF] Griggio | A practical approach to satisability modulo linear integer arithmetic[END_REF]. This goal was achieved by encoding associative properties on modular arithmetic with the help of if-then-else expressions. However, with the right preprocessing techniques, e.g., a combination of shared monomial lifting and constant if-then-else bounding (Subsection 8.2.3), these problems become almost trivial to solve. In fact, SPASS-SATT needs less than one second for each problem instance and needs only techniques for linear rational arithmetic to solve each of them.

In Figure 8.17, we see that SPASS-SATT without our preprocessing techniques solves only 136 out of the 294 rings benchmark instances and is by far slower on the instances it can solve. In Figures 8.18 & 8.19, we see that SPASS-SATT performs not much better if we just perform either shared monomial lifting or constant if-then-else bounding. However, SPASS-SATT with both shared monomial lifting and constant if-then-else bounding solves all of the 294 benchmark instances in no time. This means that only the combination of the two techniques produces an actually beneficial result. Thanks to the efficient detection of (un)bounded directions, we were now also able to design a much better bounding transformation (Chapter 6), i.e., a transformation that reduces an unbounded problem to an equisatisfiable bounded problem. Bounding transformations are interesting because most linear mixed decision procedures, e.g., branch-and-bound, become terminating on bounded problems. Bounding transformations are, therefore, extensions that complete other decision procedures. Previous bounding transformations from the literature, e.g., a priori bounds [START_REF] Papadimitriou | On the complexity of integer programming[END_REF], are inefficient with respect to actual implementations because they orient themselves only on the easiest to measure structural properties of the problem, e.g., the number of variables and the absolute size of constants. As a result, the transformed problems cannot be solved in reasonable time (e.g. the life time of earth) even for small unbounded problems. This is also the reason why previous bounding transformations cannot be found in any state-of-the-art linear arithmetic solver.

Our transformation orients itself on more significant structures: the bounded and unbounded inequalities in the input problem. To be more precise, our bounding transformation consists of two steps: First, we use the Double-Bounded reduction (Section 6.3) to eliminate all unbounded inequalities from our problem. Then we use the Mixed-Echelon-Hermite transformation (Section 6.2) to shift the variables in our problem to ones that are either bounded or do not appear in the new inequalities and are, therefore, eliminated. The result of these two steps is a bounded problem that is solvable in practice. Benchmark experiments provide further evidence to the efficiency of our bounding transformation in practice (Section 6.5). Moreover, we presented a polynomial method for converting certificates of (un)satisfiability from the transformed to the original problem. We also integrated some of our new decision procedures into our linear arithmetic theory solver SPASS-IQ (Chapter 7). To be more precise, we integrated (i) the unit cube test, (ii) our method that efficiently detects all (un)bounded directions, which is based on our methods for equality investigation, and (iii) our bounding transformation consisting of the Double-Bounded reduction and the Mixed-Echelon-Hermite transformation. Moreover, we extended our theory solver SPASS-IQ into the CDCL(LA) implementation SPASS-SATT (Chapter 8).

Our implementations also allow us to confirm the practical efficiency of our methods through various benchmark experiments (Sections 4.5 & 6.5). As a byproduct, we were also able to perform an impact analysis (i) of various other techniques from linear arithmetic for SMT solving and (ii) of the most efficient ways of implementing these methods (Chapters 7 & 8). Moreover, we participated with SPASS-SATT in the 13th International Satisfiability Modulo Theories Competition (SMT-COMP 2018) and ranked first in the category QF LIA (quantifier free linear integer arithmetic) and ranked second in the category QF LRA (quantifier free linear rational arithmetic