Tomer Libal
email: tomer.libal@inria.fr

Alexander Steen
email: a.steen@fu-berlin.de

Towards a Substitution Tree Based Index for Higher-order Resolution Theorem Provers

One of the reasons that forward search methods, like resolution, are efficient in practice is their ability to utilize many optimization techniques. One such technique is subsumption and one way of utilizing subsumption efficiently is by indexing terms using substitution trees. In this paper we describe an attempt to extend such indexes for the use of higher-order resolution theorem provers. Our attempt tries to handle two difficulties which arise when extending the indexes to higher-order. The first difficulty is the need for higher-order anti-unification. The second difficulty is the closure of clauses under associativity and commutativity. We present some techniques which attempt to solve these two problems.

Introduction

Term indexing is a popular technique for speeding up computations to a broad variety of tools in computational logic, ranging from resolution based theorem provers [START_REF] Alan | A machine-oriented logic based on the resolution principle[END_REF] to interpreters of programming languages such as Prolog. A key aspect for optimizing such tools is to use sophisticated methods for redundancy elimination. One such method for clause-based indexes is subsumption. Forward subsumption allows to discard clauses which are less general than clauses already processed, whereas backward subsumption allows to remove stored clauses if a more general one is encountered. The idea behind both is that using more general clauses drastically reduces the search space while still being enough for completing certain tasks [START_REF] Leitsch | The Resolution Calculus[END_REF]. Due to the importance of subsumption in practical applications, there is a variety of indexing techniques that support efficient subsumption queries. We refer the reader to the survey about term indexing by Ramakrishnan et al. [START_REF] Ramakrishnan | Term indexing[END_REF].

In a similar way to tools for first-order logic, higher-order logic tools can benefit also from term indexing for efficient subsumption. Unfortunately, higher-order matching, which is a required technique for determining subsumption, is much more complex than its first-order counter part [START_REF] Stirling | Decidability of higher-order matching[END_REF]. We only know of few approaches to higher-order term indexing: Theiß and Benzmüller designed a term index [START_REF] Theiss | Term indexing for the LEO-II prover[END_REF] for the LEO-II higher-order resolution theorem prover [START_REF] Benzmüller | The Higher-Order Prover LEO-II[END_REF]. However, their approach focuses on efficient low-level term (traversal) operations such as β-normalization and occur-checks. Additionally, term sharing is employed in order to reduce space consumption and for allowing constant-time equality checks between α-equivalent terms.

The closest to our approach is the higher-order term indexing technique by Pientka [START_REF] Pientka | Higher-order term indexing using substitution trees[END_REF]. Both approaches are based on substitution trees [START_REF] Graf | Substitution tree indexing[END_REF] and on propagating to the leaves those components of the indexed terms for which there is no efficient unification algorithm. One difference is the assumed term representation: In order to efficiently manage more complex type systems, such as dependent types, Pientka has chosen to represent terms using Contextual Modal Type Theory [START_REF] Nanevski | Contextual modal type theory[END_REF]. This technique allows for an elegant treatment of dependent types but requires a specially designed unification procedure [START_REF] Pientka | Optimizing higher-order pattern unification[END_REF]. Our use of the standard simple type theory [START_REF] Church | A formulation of the simple theory of types[END_REF] allows us to use type-independent unification algorithms [START_REF] Baumgartner | A variant of higher-order antiunification[END_REF]. Another difference is the technique chosen for propagating the non-pattern content: While we use an efficient algorithm for computing the pattern generalization of two non-pattern terms [START_REF] Baumgartner | A variant of higher-order antiunification[END_REF] (also made possible by our choice of term representation), Pientka's technique is based on a normalizing pre-processing step [START_REF] Pientka | Optimizing higher-order pattern unification[END_REF] which computes the non-pattern content as additional constraints and might incur an additional cost. A somewhat less crucial difference is the treatment of associativity and commutativity (AC). Since unification algorithms which incorporate this theory do not exist for higher-order logic, we deal with the problem by integrating the treatment of AC into the operations of the index. Pientka's index primary target are not terms in clausal form and therefore, this problem is not treated there. It should not be too complex, though, to integrate the ideas presented in this paper in Pientka's index in order to achieve the same AC treatment. Obvious other differences include the state of the art of Pientka's index, its implementation within the Twelf system [START_REF] Pfenning | System description: Twelfa meta-logical framework for deductive systems[END_REF], and the experimental results which are included and its rigorous representation. Our index is yet to be implemented and experimented with and we are still to provide fully rigorous presentation. Nevertheless, we believe that our approach might be more suitable for indexing higher-order terms in general higher-order theorem provers. This claim is, of course, to be justified by the implementation and experimentation of both indexing techniques within Leo-III [START_REF] Wisnieski | The Leo-III project[END_REF].

In this paper we hence present a higher-order indexing technique for the theorem prover Leo-III which is also based on substitution trees. The differences just discussed and especially the treatment of non-pattern terms might suggest that forward and backward subsumption operations can be efficiently handled by our approach.

The main difficulty which arises when trying to store clauses in an index is that the index must be closed under the AC properties of clauses. In addition, when computing subsumed clauses, the number of literals in the clause is not as important as the fact that each literal in the subsuming clauses must generalize a literal in the subsumed one. As long as we only treat unit clauses, no special treatment is required and the technique presented in [START_REF] Graf | Advanced indexing operations on substitution trees[END_REF] can be safely extended to deal with higher-order terms. When dealing with multi-literal clauses, though, one has to compensate between optimizing the size of the index and optimizing the operations over the index. As can be seen in [START_REF] Voronkov | Implementing bottom-up procedures with code trees: a case study of forward subsumption[END_REF], one cannot avoid an expensive backtracking search.

We suggest a different approach which is supposed to take advantage on searching the index in parallel. In order to achieve that, we plan to store each literal independently in the index and on subsumption calls, to retrieve and compare the literals in parallel. Due to the fact that Leo-III is based on a multi-agent architecture [START_REF] Steen | Agent-Based HOL Reasoning[END_REF], we hope that such an approach would be efficient in practice.

Since we are using substitution trees, we are still faced with the problem of using the costly higher-order unification and anti-unification procedures. In the presented work we somehow avoid this problem by using a variant of higher-order anti-unification [START_REF] Baumgartner | A variant of higher-order antiunification[END_REF] which computes pattern [START_REF] Miller | Unification of simply typed lambda-terms as logic programming[END_REF] substitutions. The use of this algorithm will allow us to maintain a substitution tree all of whose inner nodes are pattern substitutions, on which unification and anti-unification are efficient.

The definitions and properties of our index are still being investigated and some of them are not yet formally proved. Nevertheless, we hope that the arguments and examples will convince the reader about the potential of our approach for the indexing of arbitrary higher-order clauses and for the support of the forward and backward subsumption functions over this index.

In the next section we present the necessary definitions required for understanding this paper as well as the basic ideas of substitution trees and higher-order anti-unification. Following this section is the main part of the paper, in which we introduce our notion of higher-order substitution trees and define the insert, delete, retrieve and subsumption functions. We close our paper with a conclusion which also describes potential future work.

Preliminaries

In this section we present the logical language that is used throughout the paper. The language is a version of Church's simple theory of types [START_REF] Church | A formulation of the simple theory of types[END_REF] with an η-conversion rule as presented in [START_REF] Pieter | The Lambda Calculus -Its Syntax and Semantics[END_REF] and with implicit α-conversions. Unless stated otherwise, all terms are implicitly converted into β-normal and η-expanded form.

Let T 0 be a set of basic types, then the set of types T is generated by T := T 0 | T → T. Let C be a signature of function symbols and let V be a countably infinite set of variable symbols. In our definitions and examples the symbols u, w, x, y, z, W, X, Y, Z ∈ V, and f, g, h, k, a, b, c ∈ C are used. We sometimes use subscripts and superscripts as well. The set Term α of terms of type α is generated by Term α := f α | x α | (λx β .Term γ ) | (Term β→α Term β ) where f ∈ C, x ∈ V and α ∈ T (in the abstraction, α = β → γ). Applications throughout the paper will be associated to the left. We will sometimes omit brackets when the meaning is clear. We will also normally omit typing information when it is not crucial for the correctness of the results. τ (t α ) = α refers to the type of a term. The set Term denotes the set of all terms. positions are defined as usual. We denote the subterm of t at position p by t| p . Bound and free variables are defined as usual. Given a term t, we denote by hd(t) its head symbol.

Substitutions and their composition (•) are defined as usual, namely (σ • θ)(X) = θ(σ(X)). The domain and codomain of a substitution σ are denoted by dom(σ) and codom(σ). The image of σ is the set of all variables in codom(σ). We denote by σ| W the substitution obtained from substitution σ by restricting its domain to variables in W . We denote by σ[X → t] the substitution obtained from σ by mapping X to t, where X might already exist in the domain of σ. The join of two substitutions σ and θ is denoted σ • θ (cf. [START_REF] Graf | Advanced indexing operations on substitution trees[END_REF]). We extend the application of substitutions to terms in the usual way and denote it by postfix notation. Variable capture is avoided by implicitly renaming variables to fresh names upon binding. A substitution σ is more general than a substitution θ, denoted σ ≤ θ, if there is a substitution δ such that σ • δ = θ. Similarly, a substitution σ is the most specific generalization of substitutions τ and θ if σ ≤ τ , σ ≤ θ and there is no other substitution δ fulfilling these properties such that δ > σ. A substitution σ matches a substitution τ if there is a substitution δ such that δ • τ = σ. A complete set of matchers between substitutions σ and τ is a set A of substitutions such that A contains all the matching substitutions between σ and τ . A substitution σ is a renaming substitution if codom(σ) ⊆ V and |codom(σ)| = |dom(σ)|. The predicate rename(σ) is true iff σ is a renaming substitution. We denote the inverse of a renaming substitution σ by inverse(σ).

Substitution Trees

This section describes substitution trees based on those defined in [START_REF] Graf | Substitution tree indexing[END_REF]. In order to optimize some functions on trees, the definition in [START_REF] Graf | Substitution tree indexing[END_REF] uses normalized terms and substitutions. As we will see, we will insert the literals of a clause independently into the index and therefore, if we normalize them as suggested in [START_REF] Graf | Substitution tree indexing[END_REF] the relationship between the free variables among the different literals will be lost. The main differences, therefore, between our presentation and that in [START_REF] Graf | Substitution tree indexing[END_REF] is that we will avoid normalizing terms and substitutions and in addition, allow terms to be of arbitrary order. Definition 1 (Substitution Trees). A substitution tree is defined inductively and is either the empty tree or the tuple (σ, Π) where σ is a substitution and Π is a set of substitution trees such that 1. each node in the tree is either a leaf node (σ, ∅) or an inner node (σ, Π) with |Π| ≥ 2.

2. for every branch (σ 1 , Π 1 ), . . . , (σ n , Π n ) in a non-empty tree we have dom(σ i ) ∩ (dom(σ 1 ) ∪ . . . ∪ dom(σ i-1 )) = ∅ for all 0 < i ≤ n.

Higher-order Anti-unification

Anti-unification denotes the problem of finding a generalization t of two given terms t 1 and t 2 , i.e. a term t such that there exist substitutions σ 1 , σ 2 such that tσ 1 = t 1 and tσ 2 = t 2 . A key algorithm for the procedures which are described in the remainder of this paper is the higher-order anti-unification algorithm of Baumgartner et al. [START_REF] Baumgartner | A variant of higher-order antiunification[END_REF]. This algorithm differs from most higher-order unification and anti-unification procedures not only by being applicable to arbitrary (simply typed) higher-order terms, but also by efficiently computing very specific generalizations.

By very specific generalization (in contrast to the most specific one) we here mean the most specific higherorder pattern which generalizes two arbitrary higher-order terms. This pattern, however, might not be the most specific generalization of these two higher-order terms. Higher-order patterns are restricted forms of higher-order terms for which it is known that efficient unification algorithms exist [START_REF] Miller | Unification of simply typed lambda-terms as logic programming[END_REF].

Details about the higher-order pattern fragment or even the above anti-unification algorithm are not crucial for understanding this paper and are therefore omitted. It is important to note that since only most specific pattern generalizations are found, the size of the index described in this paper is not optimal. We will explain this point in more detail later.

The anti-unification algorithm of Baumgartner et al. is subsequently denoted by msg * .

Definition 2 (Algorithm msg * [START_REF] Baumgartner | A variant of higher-order antiunification[END_REF]). The algorithm msg * takes two arbitrary higher-order terms t 1 and t 2 as input and returns a higher-order pattern s and substitutions σ 1 and σ 2 such that 1. sσ 1 = t 1 and sσ 2 = t 2 , and 2. there is no other higher-order pattern s fulfilling the above property such that there is a non-trivial substitution δ where sδ = s .

Baumgartner et al. showed that the algorithm msg * computes a unique solution (up to renaming of free variables) and takes cubic time [START_REF] Baumgartner | A variant of higher-order antiunification[END_REF]. In this paper we are interested in a variant of this algorithm which computes substitutions, rather than terms and which is defined as follows:

Definition 3 (Most specific pattern generalizing substitution). Given substitutions θ 1 and θ 2 , the substitution σ is the most specific pattern generalizing substitution if there are substitutions τ 1 and τ 2 such that codom(σ) contains only higher-order patterns, σ • τ 1 = θ 1 , σ • τ 2 = θ 2 and there is no substitution σ > σ fulfilling these properties.

An algorithm for computing the most specific pattern generalizing substitution, denoted msg, can be defined on top of msg * . Definition 4 (The algorithm msg). The algorithm msg takes two substitutions θ 1 , θ 2 as input and returns a triple

(σ, τ 1 , τ 2 ) with σ, τ 1 , τ 2 as in Def. 3. To that end, let dom(θ 1 )∪dom(θ 2 ) = {x 1 , . . . , x n }. Let (f (s 1 , . . . , s n ), τ 1 , τ 2 ) = msg * (f (θ 1 (x 1 ), . . . , θ 1 (x n )), f (θ 2 (x 1 ), . . . , θ 2 (x n ))) where f is a new function symbol of arity n. Finally, set σ := {x 1 → s 1 , . . . , x n → s n }.
Claim 5. Let θ 1 , θ 2 be two substitutions and let (σ, τ 1 , τ 2 ) = msg(θ 1 , θ 2 ). Then σ is a most specific pattern generalizing substitution of θ 1 and θ 2 . Also, σ is unique up to renaming of free variables.

The algorithm msg takes cubic time, hence can be used to efficiently build up a substitution tree index (cf. next section).

Substitution Trees for Higher-order Clauses

In this section we will describe some modifications to first-order substitution trees which will allow us to extend them to higher-order terms.

The most obvious obstacle in extending the trees to higher-order terms is the fact that substitution trees depend on procedures for unification, anti-unification and matching. These procedures, while being both relatively efficient and unitary in classical first-order logic [START_REF] Martelli | An efficient unification algorithm[END_REF], are highly complex in higher-order logics and do not possess unique solutions any more [START_REF] Gérard | A unification algorithm for typed lambda-calculus[END_REF][START_REF] Snyder | Higher-order unification revisited: Complete sets of transformations[END_REF].

Another obstacle is the fact that since we are targeting resolution theorem provers, the terms we are going to store, retrieve, delete and check for subsumption are not mere syntactic terms but clauses with are closed under AC. In the first-order case, one can use dedicated unification algorithms which, although not unitary any more [START_REF] Fages | Associative-commutative unification[END_REF], are still feasible. In the higher-order case, due to the complex nature of even the syntactic unification procedure, one needs to find another approach.

Our solution to the first problem is to relax a core property of substitution trees and allow also non least general generalizations as substitutions in the nodes of the trees. Towards this end, we employ the anti-unification algorithm from section 2.2. The use of this algorithm will render our trees less optimal as nodes may now contain more general substitutions than necessary and therefore one child may be more general than another child of a node. On the other hand, the algorithm is only cubic in time complexity and is unitary.

Our approach to the second problem is to handle the AC properties of clauses not on the anti-unification or matching level, but to encode their treatment into the retrieval, insertion, deletion and subsumption functions. We obtain this by regarding each literal of a clause as an independent higher-order term and, in addition, assigning labels that are identical for all literals of the same clause.

Classical substitution trees depend on the anti-unification algorithm for treating associativity and commutativity as well as other properties required by subsumption, such as one clause being a sub-clause of the other. If such an anti-unification algorithm for higher-order term can be found, a simple extension to the trees in section 2.1 can be defined which enjoys the same definitions for insert, delete and retrieval as defined in [START_REF] Graf | Substitution tree indexing[END_REF]. This extension will also preserve the property of substitution trees that the index does not contain variants of substitutions already stored and the deletion function removes all variants of some input substitution. Since it is not clear if such an anti-unification procedure can be practical, we choose a different approach which will greatly change the form of the substitution trees. Our treatment of subsumption requires the substitution trees to contain all inserted literals, including those which are variants of already stored ones. Similarly, delete will only remove certain substitutions and not all variants of an input substitution. In addition, we will have all stored substitutions be associated with a set of labels which denotes the clause they are part of. In this way we can insert and retrieve the literals independently for a given clause.

x0 → x1 x1 → = x2 x3 x2 → a (b u) x3 → u (1) x2 → a (b u1) u2 x3 → u1 u2 (2) x1 → a (b u1) u2 (3) x1 → u1 u2 (4) x1 → ¬ x2 x2 → u3 u2 (3) x2 → a (b u3) u2 (4)
We will now give the refined definition of higher-order substitution trees.

Definition 6 (Higher-order Substitution Trees). A higher-order substitution tree is the triple (σ, Π, L) where σ is a substitution, Π is a set of substitution trees and L is a set of labels such that the following properties hold:

• if L = ∅, then Π is not empty.
• if Π is not empty, then σ is a pattern substitution.

From now on we will refer to higher-order substitution trees just as substitution trees. The above definition makes retrieval in the tree much more efficient as the traversal of the tree will always use the algorithm msg and not less efficient algorithms.

Our trees will always have a root node {x 0 → x 1 } and will therefore never be empty. This is done in order to simplify the insertion operation as the root node will always be more general than any inserted term.

Example 7. We use as running example, the manipulation of the index as done by Leo-III [START_REF] Wisnieski | The Leo-III project[END_REF] when running on a variant of Cantor's surjective theorem. The first six clauses which are inserted are the following (where α and β are types, α := ι → β and β := ι → o):

(1) = (a α (b β→ι u β )) u β (2) = (a α (b β→ι u β 1 ) u ι 2 ) (u β 1 u ι 2 ) (3) ¬ (u β 3 u ι 2 ) ∨ (a α (b β→ι u β 1 ) u ι 2 ) (4) (u β 1 u ι 2 ) ∨ ¬ (a α (b β→ι u β 3 ) u ι 2 ) (5) (a α (b β→ι u β 1 ) u ι 2 ) ∨ ¬ (a α (b β→ι u β 3 ) u ι 2 ) (6) (u β 1 u ι 2 ) ∨ ¬ (u β 3 u ι 2 )
Here, ∨ denotes the union of single literals. Note that clause (5) is subsumed by both clauses (3) and ( 4) and that clause (6) subsumes clauses (3), ( 4) and ( 5). We will use this clause set to demonstrate the insert, retrieval and delete operations of the higher-order substitution trees. Fig. 1 displays the higher-order substitution tree after the insertion of the first four clauses.

The above example demonstrates why our trees are not optimized as for example, one of the children of node x 0 → x 1 , the node x 1 → a (b u 1 ) u 2 is less general than the child x 1 → u 1 u 2 . Also, it demonstrates the problem of using arbitrary higher-order terms in the inner nodes of the tree as there are many possible substitutions which generalizes x 1 → u 1 u 2 to x 1 → a (b u 1 ) u 2 , but only one most specific pattern generalization, x 0 → x 1 .

In our trees, each branch from the root of the tree to a leaf corresponds to a literal of some clause. Since we will need to know the actual substitutions at these leaves, we introduce the notation of composed substitutions: Definition 8 (Composed Substitutions). Given a substitution tree T = (σ, Π, L) and let τ be a substitution at a leaf of T such that the nodes on the branch from the root of T to τ are (σ 1 , . . . , σ n ). Then, the composed substitution for τ , denoted comp-sub(τ ), is given by comp-sub(τ

x 0 → x 1 x 1 → f(y) (2) y → a (1) Π (a) Variant insertion x 0 → x 1 x 1 → f(x 2 ) x 2 → b (2) x 2 → a (1) Π (b) Compatible insertion x 0 → x 1 Π x 1 → f(a) (1)
) = σ 1 • . . . • σ n .
For example, in Fig. 1, consider the leftmost leaf labeled [START_REF] Pieter | The Lambda Calculus -Its Syntax and Semantics[END_REF]. Here, the composed substitution is given by

{x 0 → x 1 } • {x 1 → = x 2 x 3 } • {x 2 → a (b u), x 3 → u} = {x 0 → = a (b u) u}.
Note that, in the context of substitution trees, σ • θ equals σ • θ| dom(σ) if σ and θ are substitutions on a path from the root node to a leaf where θ occurs directly below (i.e. as a child node of) σ.

Insertion

We will define now how to insert new elements into the tree. The definition is similar to the insertion function defined in [START_REF] Graf | Substitution tree indexing[END_REF]. One difference is the use of msg for both finding variants and computing generalizations as well as adding labels to the leaves of the tree. An even more important difference is that since we store clauses and not terms, we must also store in the tree variants of existing nodes.

Given a clause labeled by l for insertion, we insert each literal t of the clause separately. In the following algorithm, we insert to the tree the substitution τ = {x 0 → t}. Definition 9 (Insertion Function insert). Let (σ, Π, L) be a substitution tree, τ a substitution to be inserted and l the clause label of this substitution. Compute the following set

A = {(θ i , δ 1 i , δ 2 i ) | (σ i , Π i , L i ) ∈ Π, (θ i , δ 1 i , δ 2 i ) = msg(σ i , τ )}.
Then, insert((σ, Π, L), τ, l) = (σ, Π , L) where:

• (Variant) if there exists (θ i , δ 1 i , δ 2 i ) ∈ A such that δ 1 i is a renaming, then Π = Π \ {(σ i , Π i , L i )} ∪ {insert((σ i , Π i , L i ), inverse(δ 1 i ) • δ 2 i , l)}.
• (Compatible) otherwise, if there exists (θ i , δ 1 i , δ 2 i ) ∈ A such that codom(θ i ) contains non-variable terms, then

Π = Π \ {(σ i , Π i , L i )} ∪ {(θ i , {(δ 1 i , Π i , L i ), (δ 2 i , ∅, {l})})}.
• (Non compatible or empty) otherwise, let (θ, δ 1 , δ 2 ) = msg(σ, τ ) and Π = Π ∪ {(δ 2 , ∅, {l})}.

Example 10. Assume we want to insert the substitution τ = {x 0 → f (a)} for clause (1) into the tree ({x 0 → x 1 }, Π).

• if Π = ∅, then we get the tree in Fig. 2c where Π is empty.

• if Π = {({x 1 → f (y)}, ∅, {(2)})} ∪ Π , then we have a variant node since msg({x 1 → f (y)}, τ ) = ({x 1 → f (
y)}, id, {y → a}) and we get the tree in Fig. 2a.

• if there is no variant but Π = {({x 1 → f (b)}, ∅, {(2)})}∪Π , then we have a compatible node since msg({x 1 → f (b)}, τ ) = ({x 1 → f (x 2 )}, {x 2 → b}, {x 2 → a}) and codom({x 1 → f (x 2 )}
) contains non variable symbols and we get the tree in Fig. 2b.

• if there is also no compatible child, we get the tree in Fig. 2c.

The way we insert substitutions into the tree preserves the tree being a substitution tree.

Claim 11. If T is a higher-order substitution tree and τ a substitution, then insert(T, τ, l) is also a higher-order substitution tree.

x 0 → x 1 x 1 → = (a (b u)) u (1)
(a) Inserting clause (1)

x 0 → x 1 x 1 → = x 2 x 3 x 2 → a (b u) x 3 → u (1) x 2 → a (b u 1 ) u 2 x 3 → u 1 u 2 (2) (b) Inserting clause (2)
Figure 3: Insertion of clauses ( 1) and ( 2) to the index from Ex. 12

x 0 → x 1 x 1 → = x 2 x 3 x 2 → a (b u) x 3 → u (1) x 2 → a (b u 1 ) u 2 x 3 → u 1 u 2 (2)
x 1 → a (b u 1 ) u 2

(3) The above property will make retrieval in the tree much more efficient as the traversal of the tree will always use the algorithm msg and not less efficient algorithms.

x 1 → ¬ (u 3 u 2 ) (3)
We show next how to insert the clauses of our running example.

Example 12. Figures 3, 4 and 1 show the state of the index after consecutive insertion of clauses ( 1) and ( 2), (3), and (4), respectively.

Note that, in the above tree, it can be seen why the tree is not optimal when using msg. Although the nodes having a (b u 1 ) u 2 are instances of u 1 u 2 , the tree does not capture it and creates two separate nodes. This happens because u 1 u 2 is not in the pattern fragment.

Deletion

While the insert function defined in the previous section did not differ much from the definition in [START_REF] Graf | Substitution tree indexing[END_REF], our definition of the deletion function is completely different. Deletion in first-order substitution trees serves as a logical operation and can be used to perform some limited backward subsumption. Since we store the literals of a clause independently in the tree, we need more information before we can decide if a substitution can be deleted. We will therefore define the deletion function as an optimization function which will remove from the index certain labels of clauses. Since such a deletion can leave some leaves of the tree without labels, we need to recursively optimize the tree by removing and merging nodes.

The formal definition of the deletion function del is given by: Definition 13 (Deletion Function del). Given a substitution tree T = (σ, Π, L) and a clause to be removed labeled by l, the function del(T, l) is defined as follows:

• Let Π = T ∈Π del(T , l).

• if σ = {x 0 → x 1 } (root) return (σ, Π , L \ {l}).
• else if Π = ∅ and L = {l}, then return .

• else if Π = {(σ , Π , L )} and |L \ {l}| = 0, then return (σ • σ , Π , L ).

• else return (σ, Π , L \ {l}).

Example 14. After deleting clauses ( 3) and ( 4) from the tree in Fig. 1, the result is the tree in Fig. 3b.

Retrieval

Retrieval in substitution trees is used in order to retrieve all substitutions with a specific relation to some input substitution. Since we are interested in forward and backward subsumption, the relations we are interested in are for the input substitution to be less and more general than the substitutions in the tree, respectively. In order to support associativity and commutativity of clauses, our substitution trees use a non-standard indexing mechanism where each literal is being stored independently from the other literals of the clause. This will prompt us, for each subsumption call, to try to retrieve all substitutions in the tree with a specific relation to all literals of an input clause. Since all retrieve calls can be done in parallel, Leo-III, with its multi-agent architecture, can take advantage on this approach to the associativity and commutativity of higher-order clauses. It should be noted that since we consider the literals of a clause separately, but a subsumption check requires a common substitution to be applicable to the clause, we need to gather, in addition to the labels, all substitutions that denote the relationship between the literals in the index and the literals of the input clause.

One property of higher-order terms cannot be avoided. In order to retrieve substitutions, a matching algorithm between two arbitrary higher-order substitutions must be used. We have avoided its use when inserting elements by using the anti-unification algorithm from Sec. 2.2. This algorithm will also allow us to traverse the tree when retrieving substitutions and reach a possible matching node according to the definition of substitution trees. The last action of the retrieval operation, the actual matching of a node with the input substitution, requires a stronger algorithm than msg. For this operation we will use a standard higher-order matching algorithm. Since the call to this algorithm is performed at most once for each stored substitution and since incomplete higher-order matching algorithms can perform very well in practice, we hope that this step will not impair much the efficiency of the trees. The use of incomplete algorithms is not essential here as a failure to match two substitutions when checking for subsumption might only increase the size of the substitution tree.

In the following, we will assume being given a (possibly incomplete) matching algorithm. Such an algorithm can be, for example, based on Huet's pre-unification procedure [START_REF] Gérard | A unification algorithm for typed lambda-calculus[END_REF] with bounds on the depth of terms.

Definition 15 (Matching Algorithm match). Given two substitutions σ and τ , then match(σ, τ ) = M where M is a complete set of matchers between σ and τ .

We now describe the two supported retrieve calls, which will both return a set of labels corresponding to an input substitution. Each label will be associated to a set of substitutions. The first of such functions returns all labels of substitutions which are more general than the input argument. Intuitively, this function traverses the tree and uses msg for checking if the input substitution is a variant of the respective node. A formal definition is given by Definition 16 (Retrieval Function g-retrieve). Given a substitution tree T = (σ, Π, L) and a substitution τ , g-retrieve(T, τ, τ ) returns a set of labels associated with substitutions, defined inductively as follows (τ = τ at the initial call, but it may change during traversal):

• if Π = ∅ and M = match(τ, comp-sub(σ)) such that M is not empty, then return (L, M ). • otherwise, return {(L, match(τ, comp-sub(σ))}∪ {g-retrieve((σ , Π , L ), τ, δ 2 ) | (σ , Π , L ) ∈ Π, msg(σ , τ ) = (θ, δ 1 , δ 2 ), rename(δ 1 )}∪ {g-retrieve((σ , ∅, L ), τ, ) | (σ , ∅, L ) ∈ Π}.
Example 17. Let the substitution tree T be that from Fig. 1 and let τ = {x 0 → (a (b u 1 ) u 2 )}. The execution of g-retrieve(T, τ ) proceeds as follows:

• apply msg on all children of the root in order to find either a variant or a leaf, and obtain the two inner nodes

({x 1 → (a (b u 1 ) u 2 )}, ∅, {(3)}) and ({x 1 → (u 1 u 2 )}, ∅, {(4)}).
• recursively apply g-retrieve on the two leaves.

• now apply match on the two composite substitutions {x 0 → (a (b u 1 ) u 2 )} and {x 0 → (u 1 u 2 )}, in order to obtain the two sets

{δ 1 = {u 1 → u 1 , u 2 → u 2 }} and {δ 2 = {u 1 → λz.(a (b u 1 ) u 2 )}, . . .}.
• for each leaf, since the result is not empty, the function returns the sets {((3), {δ 1 })} and {((4), {δ 2 , . . .})}.

Note that the set of matchers can contain more than one element and even be infinite. We hope to optimize this function in the future.

Claim 18. The set returned by g-retrieve((σ, Π, L), τ ) contains all the labels of substitutions θ which are stored in (σ, Π, L) and which are more general than τ . In addition, if a substitution δ is associated with the label of θ, then δ • θ = τ .

The second retrieval function returns all substitutions which are less general than the input substitution. The tree is still traversed using the function msg but this time we will not be able to use msg to check if σ is a variant of τ . This is due to the fact that we used msg to check if τ is a variant of σ by checking if δ 1 is a renaming substitution for msg(σ, τ ) = (θ, δ 1 , δ 2 ). This worked as both σ and θ are pattern substitutions. If we try to check whether σ is a variant of τ , since τ might not be a pattern substitution, δ 2 might not be a renaming substitution even if it is a variant. On the other hand, if σ is a variant of τ , then all the nodes in the subtrees of σ are variants of τ , so we need to use match only when we reach a node of which τ is not a variant and stop there.

We first introduce an utility function which gathers all literals on the leaves of a tree.

Definition 19 (Gathering of labels). Given a substitution tree T = (σ, Π, L) and a substitution τ , gather(T, τ

) = (L, match(comp-sub(σ), τ )) ∪ {gather(T , τ ) | T ∈ Π}.
Example 20. The application of gather(T, τ ) where T is the last child of the root in Fig. 1 and τ = {x 0 → (u 1 u 2 )} proceeds as follows:

• since there are no labels in the node, it proceed recursively on the two children.

• the application on the first child returns {((3), A)}, where A = {u 1 → λz.¬(u 3 u 2 ), . . .} is the result of match on the composite function {x 0 → ¬(u 3 u 2 )} and τ .

• the application on the second child returns {((4), A)}, where A = {{u 1 → λz.¬(a (b u 3 ) u 2 )}, . . .} is the result of match on the composite function {x 0 → ¬(a (b u 3 ) u 2 )} and τ .

• return the union of these two sets.

Definition 21 (Retrieval Function i-retrieve). Given a substitution tree T = (σ, Π, L) and a substitution τ , i-retrieve(T, τ, τ ) returns a set of labels associated with substitutions, defined inductively as follows (τ = τ at the initial call, but it may change during traversal):

• if msg(σ, τ ) = (θ, δ 1 , δ 2 ) such that either δ 1 is not a renaming or δ 2 is a renaming, and match(comp-sub(σ), τ ) = M such that M is not empty, then return (L, M ) ∪ {gather(T, τ ) | T ∈ Π}.

• otherwise, if δ 1 is a renaming, return {i-retrieve(T , τ, δ 2 ) | (T ) ∈ Π}.

• otherwise, return ∅.

Example 22. Let the substitution tree T be the one from Fig. 1 and τ be the substitution {x 0 → (u 1 u 2 )}. The function i-retrieve(T, τ ) proceeds as follows:

• we first calculate msg({x 0 → x 1 }, τ ) = (x 0 → x 1 , id, {x 1 → (u 1 u 2 )}).
• since id is a renaming, we recursively apply i-retrieve on all nodes.

• for all children, the first case of i-retrieve now holds and we start gathering all the labels in the tree.

For brevity, we give the example only for the last node:

• as mentioned, the first case now holds for this node as msg({x

1 → ¬x 2 }, {x 1 → (u 1 u 2 )}) = ({x 1 → x 3 }, {x 3 → ¬x 2 }, {x 3 → (u 1 u 2 )})
and therefore δ 1 is not a renaming. On the other hand, match({x 0 → ¬x 2 }, {x 0 → (u 1 u 2 )}) = {{u 1 → λz.¬x 2 }, . . .} is not empty.

• gather all labels on this node, as shown in Ex. 20.

Claim 23.

The set returned by i-retrieve((σ, Π, L), τ ) contains all the labels of substitutions which are stored in (σ, Π, L) and which are less general than τ . In addition, if a substitution δ is associated with the label of θ, then δ • τ = θ.

We now show how the checks for forward and backward subsumptions of clauses can be implemented using g-retrieve and i-retrieve.

Forward Subsumption

Forward subsumption checks if an input clause is subsumed by a clause in the index. In case the clause is subsumed, no change to the index is made and the input clause is not inserted into the index.

Definition 24 (Subsumption). A clause c is subsumed by a clause d if |c| ≤ |d| and there is a substitution σ such that for each literal l of d there is a literal l of c such that lσ = l . Here, |c| denotes the number of literals in c.

Our way of treating associativity and commutativity means that for each literal of the input clause we gather all more general literals in the index with their associated substitution sets. These sets contains all substitutions which independently match the literals in the tree to the literals of the input clause. In order to detect that the input clause is subsumed by the index, we need to show two things. We first need to show that for all the literals of a clause, a more general literal is returned by the index. In addition, we need to show that for each such literal is associated a "compatible" substitution. Two substitutions are "compatible" if they agree on all variables in the intersection of their domains. This requirement means that if d is a clause in the index that subsumes the input clause c, then if each of the literals of d subsumes a literal of c with a "compatible" substitution, then there is indeed a substitution σ such that each of the literals of d subsumes a literal of c with σ.

It should be noted that the above technique is based on set subsumption, in contrast to multiset subsumption which is commonly used. A main reason for preferring multisets over sets, is to prevent cases where a clause is being subsumed by a larger clause, which is possible according to the above definition and leads to an incomplete search procedure. In order to restrict such cases, we will only allow subsumption of clasues which are not smaller than the subsuming clauses.

We will now give the formal definition of forward subsumption.

Definition 25 (Forward Subsumption fsum). Let T be a substitution tree and c = l 1 ∨ . . . ∨ l n a clause. Let A = {g-retrieve(T, {x 0 → l i } | 0 < i ≤ n}. The function fsum(T, c) returns true if there are in A the labels (l, M 1 ), . . . , (l, M k ) such that k = |l| is the number of literals of clause l, k ≤ n, and for each two sets M i and M j for 0 < i < j ≤ k, there are substitutions σ i ∈ M i and σ j ∈ M j such that σ i (x) = σ j (x) for all x ∈ dom(σ i ) ∩ dom(σ j ).

Example 26. We will follow the computation of fsum(T, c) where T is the substitution tree of Fig. 1 and c is clause (5) from Ex. 7.

• We first compute the set A which is the union of g-retrieve(T, {x 0 → (a (b u 1 ) u 2 )}) and g-retrieve(T, {x 0 → ¬(a (b u 3 ) u 2 )}).

• the first set was already computed in Ex. 17 and resulted in {((3), {{u

1 → u 1 , u 2 → u 2 }}), ((4), {{u 1 → λz.(a (b u 1 ) u 2 )}, . . .})}.
• the second set is computed in a similar way and results in {((4), {{u

3 → u 3 , u 2 → u 2 }}), ( (3)
, {{u 3 → λz.(a (b u 3 ) u 2 )}, . . .})}.
• since the size of both (3) and ( 4) is 2 and the number of occurrences of the labels of each is 2, we are just left with checking if the matching substitutions are compatible, which is easily verified since their domains are disjoint.

• we return that ( 5) is subsumed by the index (by both (3) and ( 4)).

Claim 27. If fsum(T, c) returns true, then there is a clause d indexed by T such that c is subsumed by d.

x 0 → x 1 x 1 → = x 2 x 3 x 2 → a (b u) x 3 → u (1) x 2 → a (b u 1 ) u 2 x 3 → u 1 u 2 (2) x 1 → u 1 u 2 (6) x 1 → ¬ (u 3 u 2 ) (6)
Figure 5: The final substitution tree

Backward Subsumption

Similarly to the way we treated forward subsumption, we can also define backward subsumption. When we detect that the index contains clauses which are subsumed by an input clause, we will need to modify the index in order to delete these clauses.

In order to detect all clauses in the index which are subsumed by the input clause, we need to show that the sets returned for each literal of the input clause contain the labels of the subsumed clauses and that the substitution sets associated with the same labels across sets contain "compatible" substitutions.

Definition 28 (Backward Subsumption bsum). Let T be a substitution tree and c = l 1 ∨ . . . ∨ l n a clause. Let A i = i-retrieve(T, {x 0 → l i } for all 0 < i ≤ n. Label l ∈ bsum(T, c) if n ≤ |l| and (l, M i ) ∈ A i for all 0 < i ≤ n such that for each two sets M i and M j for 0 < i < j ≤ n, there are substitutions σ i ∈ M i and σ j ∈ M j such that σ i (x) = σ j (x) for all x ∈ dom(σ i ) ∩ dom(σ j ).

Example 29. We will follow the computation of bsum(T, c) where T is the substitution tree of Fig. 1 andc is clause (6) from Ex. 7.

• We first compute the sets A 1 = i-retrieve(T, {x 0 → (u 1 u 2 )}) and A 2 = i-retrieve(T, {x 0 → ¬(u 3 u 2 )}).

• the set A 1 was already computed in Ex.

, {{u 1 → λz.¬(u 3 u 2 )}, . . .}), ((4), {{u 1 → λz.¬(a (b u 3 ) u 2 )}, . . .})}.

• the set A 2 can be computed in a similar way and results in {((3), {{u 3 → λz.(u 3 u 2 )}, . . .}), ((4), {{u 3 → λz.(a (b u 3 ) u 2 )}}), . . .}.

• we notice that only the labels (3) and (4) occur in both sets A 1 and A 2 and that each of the substitutions from A 2 can be matched with two substitutions in A 1 .

• both options are compatible, the first option means that each literals of clause (6) subsumes a different literal of clauses ( 3) and (4) while the second option means that they subsumes the same literal in these clauses.

• we conclude that they are both redundant.

• the resulted tree after their removal was computed in Ex. 14.

Claim 30. If l ∈ bsum(T, c) and the clause d is labeled by l, then d is subsumed by c.

Example 31. The result of inserting clause [START_REF] Graf | Substitution tree indexing[END_REF] to the substitution tree from the previous example can be seen in Fig. 5.

In this work, we have presented an indexing data structure for higher-order clause subsumption based on substitution trees. We make use of an efficient higher-order pattern anti-unification algorithm for calculting meaningful generalizations of two arbitrary higher-order terms. The proposed indexing method is rather limited as is does not support subsumption testing modulo associativity and commutativity which is, of course, essential for general clause subsumption. However, even the limited approach admits effective subsumption queries in certain cases. Additionally, improvements for including such AC aspects are sketched. While the index is not size-optimal in general, we believe that the approach performs quite good in practice, especially when combined with further, orthogonal indexing techniques that could be used as a pre-test. One suitable candidate is a higher-order variant of feature vector indexing [START_REF] Schulz | Simple and efficient clause subsumption with feature vector indexing[END_REF].

The substitution tree index as described here is planned for implementation in the Leo-III prover [START_REF] Wisnieski | The Leo-III project[END_REF]. We hope that due to Leo-III's agent-based architecture [START_REF] Steen | Agent-Based HOL Reasoning[END_REF], independent agents can traverse the substitution tree index in parallel. The index is mainly devised for employment in resolution-based provers, but it seems possible to generalize the approach to non-clausal-based deduction procedures.

For further work we need to investigate means of suitably enhancing msg to handle AC properties and other subsumption properties. Also, the matching algorithm could be improved such that it returns minimal complete sets of substitutions which can then be used for the subsumption procedure. At the current state of the index data structure, the inserted substitutions are not normalized. This is essentially a shortcoming that originates from the way we relate the matching substitutions of different literals of the same clause to each other. This results, however, in a substitution tree that contains occurrences of substitutions which are equivalent up to free variables renaming. To overcome this shortcoming, we need to find a way of keeping the substitutions normalized while still being able to relate the matchers of different literals.

Figure 1 :

 1 Figure 1: An example of a substitution tree

  (c) Non-compatible (or empty) insertion

Figure 2 :

 2 Figure 2: Cases of inserting a substitution into the index from Ex. 10.

Figure 4 :

 4 Figure 4: Insertion of clause (3) from Ex. 12

  22 and results in {((1), {{u 1 → λz. = (a (b u))u}, . . .}), ((2), {{u 1 → λz. = (a (b u 1 ) u 2 )(u 1 u 2 )}, . . .}), ((3), {{u 1 → λz.(a (b u 1 ) u 2 )}, . . .}), ((4), {{u 1 → λz.(u 1 u 2 ))}, . . .}), (

Acknowledgements

Work of the first author was funded by the ERC Advanced Grant ProofCert. The second author has been supported by the DFG under grant BE 2501/11-1 (Leo-III). We thank the reviewers for the very valuable feedback they provided.