Xavier Rival
email: rival@di.ens.fr

Wonyeol Lee
email: wonyeol@cs.stanford.edu

Hongseok Yang
email: hongseok.yang@kaist.ac.kr.

Keywords: • Software and its engineering → Correctness, Automated static analysis, • Mathematics of computing → Bayesian computation, Variational methods smoothness, static analysis, probabilistic programming, variational inference

We present a static analysis for discovering differentiable or more generally smooth parts of a given probabilistic program, and show how the analysis can be used to improve the pathwise gradient estimator, one of the most popular methods for posterior inference and model learning. Our improvement increases the scope of the estimator from differentiable models to non-differentiable ones without requiring manual intervention of the user; the improved estimator automatically identifies differentiable parts of a given probabilistic program using our static analysis, and applies the pathwise gradient estimator to the identified parts while using a more general but less efficient estimator, called score estimator, for the rest of the program. Our analysis has a surprisingly subtle soundness argument, partly due to the misbehaviours of some target smoothness properties when viewed from the perspective of program analysis designers. For instance, some smoothness properties, such as partial differentiability and partial continuity, are not preserved by function composition, and this makes it difficult to analyse sequential composition soundly without heavily sacrificing precision. We formulate five assumptions on a target smoothness property, prove the soundness of our analysis under those assumptions, and show that our leading examples satisfy these assumptions. We also show that by using information from our analysis instantiated for differentiability, our improved gradient estimator satisfies an important differentiability requirement and thus computes the correct estimate on average (i.e., returns an unbiased estimate) under a regularity condition. Our experiments with representative probabilistic programs in the Pyro language show that our static analysis is capable of identifying smooth parts of those programs accurately, and making our improved pathwise gradient estimator exploit all the opportunities for high performance in those programs.

CCS Concepts:

Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference

WONYEOL LEE, Stanford University, USA XAVIER RIVAL, INRIA Paris, France and ENS, CNRS, and PSL University, Paris, France HONGSEOK YANG, KAIST, South Korea and Institute for Basic Science (IBS), South Korea

INTRODUCTION

Probabilistic programs define models from machine learning and statistics, and are used to analyse datasets from a wide range of applications [START_REF] Bingham | Pyro: Deep Universal Probabilistic Programming[END_REF][START_REF] Carpenter | Stan: A Probabilistic Programming Language[END_REF][START_REF] Ge | Turing: A Language for Flexible Probabilistic Inference[END_REF][START_REF] Gehr | PSI: Exact Symbolic Inference for Probabilistic Programs[END_REF][START_REF] Goodman | Church: a language for generative models[END_REF][START_REF] Gordon | Tabular: A Schema-driven Probabilistic Programming Language[END_REF]Mansinghka et al. 2014;[START_REF] Minka | Infer.NET 2.6[END_REF][START_REF] Narayanan | Probabilistic inference by program transformation in Hakaru (system description)[END_REF][START_REF] Salvatier | Probabilistic programming in Python using PyMC3[END_REF][START_REF] Siddharth | Learning Disentangled Representations with Semi-Supervised Deep Generative Models[END_REF][START_REF] Tolpin | Design and Implementation of Probabilistic Programming Language Anglican[END_REF][START_REF] Tran | Simple, Distributed, and Accelerated Probabilistic Programming[END_REF][START_REF] Tran | Edward: A library for probabilistic modeling, inference, and criticism[END_REF][START_REF] Wood | A New Approach to Probabilistic Programming Inference[END_REF]]. These programs are written in languages with special runtimes, called inference engines, which can be used to answer probabilistic queries, such as posterior inference and marginal likelihood estimation, or to learn model parameters in those programs, such as weights of neural networks. Whether a probabilistic program is useful for, for instance, discovering a hidden pattern in a given dataset or making an accurate prediction largely lies in these inference engines. These engines should compute accurate approximations or good model parameters within a fixed time budget. It is, thus, not surprising that substantial research efforts have been made to develop efficient inference algorithms and their implementations (as inference engines) [START_REF] Tejasvi Chaganty | Efficiently Sampling Probabilistic Programs via Program Analysis[END_REF][START_REF] Holtzen | Scaling exact inference for discrete probabilistic programs[END_REF][START_REF] Kucukelbir | Automatic Variational Inference in Stan[END_REF]Mansinghka et al. 2014;[START_REF] Nori | R2: An Efficient MCMC Sampler for Probabilistic Programs[END_REF]Ritchie et al. 2016a;[START_REF] Schulman | Gradient Estimation Using Stochastic Computation Graphs[END_REF][START_REF] Van De Meent | An Introduction to Probabilistic Programming[END_REF]Wingate et al. 2011a,b;[START_REF] Wingate | Automated Variational Inference in Probabilistic Programming[END_REF][START_REF] Zhou | Divide, Conquer, and Combine: a New Inference Strategy for Probabilistic Programs with Stochastic Support[END_REF].

We are concerned with smoothness 1 properties of probabilistic programs, which have been exploited by performant posterior-inference and model-learning algorithms and engines. For instance, when probabilistic programs are differentiable (in the sense that they define differentiable unnormalised densities), their posteriors can be inferred by Hamiltonian Monte Carlo [Neal 2011], one of the best performing MCMC algorithms. Also, in that case, their posteriors and model parameters can be inferred or learnt using the pathwise gradient estimator [START_REF] Diederik | Auto-Encoding Variational Bayes[END_REF][START_REF] Jimenez Rezende | Stochastic Backpropagation and Approximate Inference in Deep Generative Models[END_REF], a popular technique for estimating the gradient of a function using samples. We also point out that the need for smoothness arises in a broader context of machine learning and computer science; Lipschitz continuity is one of the desired or at least recommended properties for neural networks [START_REF] Arjovsky | Wasserstein Generative Adversarial Networks[END_REF][START_REF] Kim | The Lipschitz Constant of Self-Attention[END_REF], and also differentiability commonly features as a requirement for pieces of code inside simulation software and cyber physical systems, where differential equations are used to specify the environment [START_REF] Platzer | Logical Foundations of Cyber-Physical Systems[END_REF]].

We present a static analysis that enables optimised posterior inference and model learning for probabilistic programs. We develop a static analysis that discovers differentiable or more generally smooth parts of given probabilistic programs, and show how the analysis can be used to improve the pathwise gradient estimator. Our improvement increases the scope of the estimator from differentiable to non-differentiable models, without requiring any intervention from the user; the improved estimator automatically identifies differentiable parts of probabilistic programs using our static analysis, and applies the pathwise gradient estimator to the identified parts while using a more general but less efficient estimator, called score estimator [START_REF] Ranganath | Black Box Variational Inference[END_REF][START_REF] Williams | Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning[END_REF], for the rest of the programs.

Our static analysis for smoothness has a surprisingly subtle soundness argument, partly due to the misbehaviours of some target smoothness properties when viewed from the perspective of program analysis designers. For instance, some smoothness properties, such as partial differentiability and partial continuity, are not preserved by function composition, and this makes it difficult to analyse sequential composition soundly without heavily sacrificing precision. In fact, overlooking such misbehaviours has been a source of errors in published static analyses for continuity [START_REF] Chaudhuri | Continuity analysis of programs[END_REF][START_REF] Chaudhuri | Continuity and robustness of programs[END_REF]. 2 We formulate five assumptions that clearly identify what a smoothness property should satisfy in order to avoid unsound analysis. Interestingly, these assumptions also determine what the property is allowed to violate. For instance, they reveal that the smoothness property does not have to be closed under the limits of chains of smooth (partial) functions, although the closure under such limits, called admissibility, has often been used to justify proof rules about or static analysis of loops. Dispensing with the admissibility requirement broadens the scope of our program analysis non-trivially; some useful smoothness properties from mathematics fail to meet the requirement.

Our variant of the pathwise gradient estimator works by program transformation and non-standard execution. It first transforms given probabilistic programs based on the results of our static analysis. Then, our estimator executes the transformed programs according to a standard (sampling) semantics, and collects sampled values during the execution. Finally, using the collected values, the estimator executes the original programs according to a non-standard (density) semantics this time. During the execution, our estimator computes a quantity involving differentiation, which becomes the estimate of the target gradient. We prove that our estimator satisfies an important differentiability requirement and thus, under a regularity condition, it is correct: the computed estimate is unbiased, i.e., it is the target gradient when averaged over random choices made during execution.

Our static analysis and variant of the pathwise gradient estimator have been implemented for a subset of the Pyro probabilistic programming language [START_REF] Bingham | Pyro: Deep Universal Probabilistic Programming[END_REF]. They have been successfully applied to the 13 representative Pyro examples, which include advanced models with deep neural networks, such as attend-infer-repeat [START_REF] Ali Eslami | Attend, Infer, Repeat: Fast Scene Understanding with Generative Models[END_REF]] and single-cell annotation using variational inference [START_REF] Xu | Probabilistic harmonization and annotation of single-cell transcriptomics data with deep generative models[END_REF]. For each of these examples, Pyro provides a (default) selective use of the pathwise gradient estimator but without any correctness guarantee. Our analysis and improved estimator automatically reproduced those uses, and proved that in those use cases, the estimator satisfies an important differentiability requirement and it is, thus, correct (i.e., unbiased) under a regularity condition (which needs to be discharged separately).

We summarise the main contributions of the paper:

• We present a program analysis for smoothness properties such as differentiability, and explain a subtle soundness argument for the analysis. Our argument identifies five assumptions for target smoothness properties, which are violated by some well-known smoothness properties and can help detect and prevent soundness errors in static analyses for smoothness properties ( §5). • We present a gradient estimator for probabilistic programs that improves the well-known pathwise gradient estimator using our program analysis. We also prove that our estimator satisfies an important differentiability requirement and it is, thus, correct (namely unbiased) under a regularity condition ( §4 and §6). • We show that our program analysis and gradient estimator can be successfully applied to representative probabilistic programs in Pyro, and can prove that existing unproved optimisations for these programs satisfy the differentiability requirement ( §7).

The appendix (i.e., §A- §I) includes omitted proofs and details, and can be found in [Lee et al. 2022b].

INFORMAL DESCRIPTION OF BASIC CONCEPTS AND OUR APPROACH

We start by describing informally basic concepts and the goal of our approach, which we hope helps the reader to see the big picture of our technical contributions. To simplify presentation, we use toy examples in the section. But we emphasise that our approach has been applied to representative Pyro programs that describe advanced machine-learning models with deep neural networks.

Probabilistic programming and variational inference. In a probabilistic programming language (PPL), a program expresses a probabilistic model. As an example, consider the program 𝑐 𝑚 in Fig. 1, which describes a probabilistic model of the random variables 𝑧 1 and 𝑧 2 in R by specifying their unnormalised density 𝑝 𝑐 𝑚 (𝑧 1 , 𝑧 2 ) = N (𝑧 1 ; 0, 5) • N (𝑧 2 ; 𝑧 1 , 3) • (1 [𝑧 2 >0] • N (0; 1, 1) + 1 [𝑧 2 ≤0] • N (0; -2, 1)),

𝑐 𝑚 =
𝑥 1 := sam("z 1 ", dist N (0, 5), 𝜆𝑦.𝑦); 𝑥 2 := sam("z 2 ", dist N (𝑥 1 , 3), 𝜆𝑦.𝑦); if (𝑥 2 > 0) {obs(dist N (1, 1), 0)} else {obs(dist N (-2, 1), 0)} , 𝑐 𝑔 = 𝑥 1 := sam("z 1 ", dist N (𝜃 1 , 1), 𝜆𝑦.𝑦); 𝑥 2 := sam("z 2 ", dist N (𝜃 2 , 1), 𝜆𝑦.𝑦) .

Fig. 1. A model 𝑐 𝑚 and a guide 𝑐 𝑔 in a PPL. Here dist N (𝑎, 𝑏) is the distribution expression, and denotes the normal distribution with mean 𝑎 and variance 𝑏.

𝑐 ′ 𝑔 =

𝑥 1 := sam("z 1 ", dist N (0, 1) , 𝜆𝑦.𝑦 + 𝜃 1 ); 𝑥 2 := sam("z 2 ", dist N (0, 1) , 𝜆𝑦.𝑦 + 𝜃 2 )

, 𝑐 ′′ 𝑔 =

𝑥 1 := sam("z 1 ", dist N (0, 1) , 𝜆𝑦.𝑦 + 𝜃 1 ); 𝑥 2 := sam("z 2 ", dist N (𝜃 2 , 1), 𝜆𝑦.𝑦) .

Fig. 2. A fully (or selectively) reparameterised guide 𝑐 ′ 𝑔 (or 𝑐 ′′ 𝑔 ).

where N (𝑥; 𝑎, 𝑏) is the probability density of a normal distribution with mean 𝑎 and variance 𝑏, and 1 [𝜑 ] is the indicator function that returns 1 if 𝜑 holds and 0 otherwise. The first two N factors in the equation come from the sample commands (sam) in 𝑐 𝑚 . They are called prior distributions, and describe prior knowledge on two random variables named 𝑧 1 and 𝑧 2 . The last factor comes from the if and observe commands (if and obs), which express that an unnamed random variable is sampled and observed to be 0 and its distribution is dist N (1, 1) or dist N (-2, 1) depending on whether 𝑧 2 is positive or not. This factor is called likelihood, and it states information about 𝑧 1 and 𝑧 2 that comes from an observed data point 0. Ignore the third arguments of the sample commands of 𝑐 𝑚 for now, which have no effect on 𝑝 𝑐 𝑚 ; they will be explained later.

The purpose of writing 𝑐 𝑚 in a PPL, called model, is to infer its normalised probability density

𝑝 𝑐 𝑚 (𝑧 1 , 𝑧 2 ) ≜ 𝑝 𝑐 𝑚 (𝑧 1 , 𝑧 2 )/ ∫ 𝑝 𝑐 𝑚 (𝑧 1 , 𝑧 2 ) 𝑑𝑧 1 𝑑𝑧 2 ,
also called normalised posterior density. Intuitively, this normalised density brings together two types of information about 𝑧 1 and 𝑧 2 , the first from their prior distributions (expressed in the first and second lines of 𝑐 𝑚 ), and the second from the observed data point 0 that depends on 𝑧 1 and 𝑧 2 (the third and fourth lines of 𝑐 𝑚 ). This inference task is called posterior inference problem. Among a wide range of approaches to the problem, we focus on the approach called variational inference, which forms the core of the recent combination of PPLs and deep learning.

In variational inference, we posit another program 𝑐 𝑔 , called guide, that is simpler than 𝑐 𝑚 and parameterised by 𝜃 . Then, we approximate the normalised density of 𝑐 𝑚 by 𝑐 𝑔 with an optimal choice of 𝜃 . For instance, consider the program 𝑐 𝑔 in Fig. 1. The program specifies the following already-normalised probability density 𝑝 𝑐 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ) = N (𝑧 1 ; 𝜃 1 , 1) • N (𝑧 2 ; 𝜃 2 , 1).

It can serve as a guide program for 𝑝 𝑐 𝑚 . To best approximate 𝑝 𝑐 𝑚 by 𝑝 𝑐 𝑔 ,𝜃 , variational inference aims at finding 𝜃 that minimises some notion of the discrepancy (called KL divergence) between 𝑝 𝑐 𝑔 ,𝜃 and 𝑝 𝑐 𝑚 , or equivalently that maximises the objective function L (called evidence lower bound):

arg max 𝜃 L (𝜃 ) for L (𝜃 ) ≜ E 𝑝 𝑐𝑔,𝜃 (𝑧 1 ,𝑧 2 ) [𝑓 𝜃 (𝑧 1 , 𝑧 2 )] with 𝑓 𝜃 (𝑧 1 , 𝑧 2 ) ≜ log(𝑝 𝑐 𝑚 (𝑧 1 , 𝑧 2 )/𝑝 𝑐 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 )).
A standard way to solve this optimisation problem is to apply the gradient-ascent algorithm: starting from an initial value 𝜃 (0) of 𝜃 , compute 𝜃 (𝑡 ) iteratively by 𝜃 (𝑡 +1) ≜ 𝜃 (𝑡 ) + 𝜂 • ∇ 𝜃 L (𝜃 (𝑡 ) ), and return 𝜃 (𝑇 ) for a sufficiently large 𝑇 ∈ N. Here 𝜂 ∈ R >0 denotes a learning rate.

A challenging part in the algorithm is to compute ∇ 𝜃 L (𝜃 ). An exact computation of the gradient is mostly intractable due to the expectation inside L, which hinders the gradient from having a closed-form formula. Hence, in practice, we rather estimate (not exactly compute) the gradient via a Monte Carlo method: draw a random sample ( ẑ1 , ẑ2 ) from some distribution 𝑞 𝜃 , apply some function 𝑔 𝜃 to the sample, and use the result as an estimate to the gradient, i.e., 𝑔 𝜃 ( ẑ1 , ẑ2 ) ≈ ∇ 𝜃 L (𝜃 ) for a sample ( ẑ1 , ẑ2 ) drawn from 𝑞 𝜃 ( ẑ1 , ẑ2 ).

(1)

An important desired property of such a gradient estimator is unbiasedness, which states that the estimate is accurate in expectation: E 𝑞 𝜃 (𝑧 1 ,𝑧 2 ) [𝑔 𝜃 (𝑧 1 , 𝑧 2 )] = ∇ 𝜃 L (𝜃 ). The property is necessary for the algorithm to converge to a local optimum, and is, thus, desired.

Gradient estimators for variational inference: SCE, PGE, and SPGE. A standard estimator for ∇ 𝜃 L (𝜃 ) is the score estimator (SCE) [START_REF] Ranganath | Black Box Variational Inference[END_REF][START_REF] Williams | Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning[END_REF], which is unbiased under mild conditions. It estimates ∇ 𝜃 L (𝜃 ) by using the recipe in Eq. ( 1) with 𝑞 𝜃 (𝑧 1 , 𝑧 2 ) = 𝑝 𝑐 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ) and

𝑔 𝜃 (𝑧 1 , 𝑧 2 ) = 𝑓 𝜃 (𝑧 1 , 𝑧 2 ) • ∇ 𝜃 log 𝑞 𝜃 (𝑧 1 , 𝑧 2 ).
That is, the estimator draws a sample from the guide distribution 𝑝 𝑐 𝑔 ,𝜃 and applies the above 𝑔 𝜃 to obtain a gradient estimate. 3 It is applicable to a wide range of model-guide pairs while remaining unbiased, but it is known to have a large approximation error (i.e., have a large variance).

The pathwise gradient estimator (PGE) [START_REF] Diederik | Auto-Encoding Variational Bayes[END_REF][START_REF] Jimenez Rezende | Stochastic Backpropagation and Approximate Inference in Deep Generative Models[END_REF]] is another standard gradient estimator, which is known to have a smaller approximation error than the SCE and thus has been a preferred option against the SCE. The PGE requires an additional program 𝑐 ′ 𝑔 that is a 𝜃 -independent reparameterisation of the guide 𝑐 𝑔 . A program 𝑐 ′ is said to be 𝜃 -independent if the probability densities of the sampled random variables in 𝑐 ′ are 𝜃 -independent. It is called a reparameterisation of 𝑐 if 𝑐 and 𝑐 ′ sample the same set of random variables and they have the same semantics on those variables in the following sense: when there are 𝑛 random variables, for any measurable ℎ : R 𝑛 → R, we have E 𝑝 𝑐 (𝑧) [ℎ(𝑣 𝑐 (𝑧))] = E 𝑝 𝑐 ′ (𝑧) [ℎ(𝑣 𝑐 ′ (𝑧))], where 𝑝 𝑐 : R 𝑛 → R is the probability density of all 𝑛 random variables in 𝑐, and 𝑣 𝑐 : R 𝑛 → R 𝑛 is the so called value function of 𝑐, which applies the lambda functions in the third arguments of 𝑐's sample commands to the corresponding random variables. For example, 𝑐 ′ 𝑔 in Fig. 2 is a 𝜃 -independent reparameterisation of 𝑐 𝑔 for 𝜃 = (𝜃 1 , 𝜃 2 ). It has the probability density 𝑝 𝑐 ′ 𝑔 (𝑧 1 , 𝑧 2 ) = N (𝑧 1 ; 0, 1) • N (𝑧 2 ; 0, 1), and the value function 𝑣 𝑐 ′ 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ) = (𝑧 1 + 𝜃 1 , 𝑧 2 + 𝜃 2 ). Note that 𝑝 𝑐 ′ 𝑔 does not depend on 𝜃 , as required by the 𝜃independence of 𝑐 𝑔 . We can show this 𝑐 ′ 𝑔 is a reparameterisation of 𝑐 𝑔 in Fig. 1 by using the fact that 𝑣 𝑐 𝑔 is the identity function and 𝑦 = 𝑥 + 𝑎 for 𝑥 drawn from N (𝑥; 0, 1) follows the distribution N (𝑦; 𝑎, 1).

Given a reparameterised guide 𝑐 ′ 𝑔 , the PGE estimates ∇ 𝜃 L (𝜃 ) by again following the recipe in Eq. (1) this time with 𝑞 ′ (𝑧 1 , 𝑧 2 ) = 𝑝 𝑐 ′ 𝑔 (𝑧 1 , 𝑧 2 ) and 𝑔 ′ 𝜃 (𝑧 1 , 𝑧 2 ) = ∇ 𝜃 𝑓 𝜃 (𝑧 ′ 1 , 𝑧 ′ 2 ) for (𝑧 ′ 1 , 𝑧 ′ 2 ) = 𝑣 𝑐 ′ 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ). This estimator differs from the SCE in two aspects. First, a random sample is drawn from a reparameterised-guide distribution 𝑝 𝑐 ′ 𝑔 , not from 𝑝 𝑐 𝑔 ,𝜃 . Next, the estimation function 𝑔 ′ 𝜃 computes the derivative of 𝑓 𝜃 (𝑧 ′ 1 , 𝑧 ′ 2 ) with respect to 𝜃 and (𝑧 ′ 1 , 𝑧 ′ 2 ) (not with respect to 𝜃 only), since the argument of 𝑓 𝜃 (-) in 𝑔 ′ 𝜃 depends on 𝜃 via 𝑣 𝑐 ′ 𝑔 ,𝜃 . 4 While having a small approximation error, the PGE requires more than the SCE to ensure the unbiasedness. An important additional requirement for the PGE is that (i) 𝑝 𝑐 𝑚 (𝑧 1 , 𝑧 2 ) and 𝑝 𝑐 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ) should be differentiable in 𝜃 and 𝑧 1 , 𝑧 2 and (ii) 𝑣 𝑐 ′ 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ) be differentiable in 𝜃 for all 𝑧 1 , 𝑧 2 . The requirement is imposed partly to ensure that no differentiation error arises in computing 𝑔 ′ 𝜃 . This differentiability requirement, however, can be easily violated if a model or a guide starts to use branches or loops. For instance, it is violated by our example in Figs. 1 and2 as 𝑝 𝑐 𝑚 (𝑧 1 , 𝑧 2 ) is not differentiable in 𝑧 2 . This violation makes the PGE biased for the example, i.e.,

E 𝑞 ′ 𝜃 (𝑧 1 ,𝑧 2 ) [𝑔 ′ 𝜃 (𝑧 1 , 𝑧 2 )] = (• • • , 1 3 (𝜃 1 -𝜃 2 )) ≠ (• • • , 1 3 (𝜃 1 -𝜃 2 ) + 3
2 N (-𝜃 2 ; 0, 1)) = ∇ 𝜃 L (𝜃 ), 3 The log term in 𝑔 𝜃 comes from the well-known log-derivative trick: ∇ 𝜃 𝑞 𝜃 (𝑧 1 , 𝑧 2 ) = 𝑞 𝜃 (𝑧 1 , 𝑧 2 ) • ∇ 𝜃 log 𝑞 𝜃 (𝑧 1 , 𝑧 2 ). 4 By the chain rule, we have the following for each 𝑖 ∈ {1, 2}:

𝜕𝑓 𝜃 (𝑧 ′ 1 , 𝑧 ′ 2 ) 𝜕𝜃 𝑖 = 𝜕𝑓 𝜃 (𝑥 1 , 𝑥 2 ) 𝜕𝜃 𝑖 (𝑥 1 ,𝑥 2 ,𝜃 ) =(𝑧 ′ 1 ,𝑧 ′ 2 ,𝜃 )
and thus causes the gradient-ascent algorithm to converge to a suboptimal 𝜃 : applying the PGE to the example produces a suboptimal solution 𝜃 = (0, 0), whereas the optimal solution is 𝜃 ≈ (0.95, 1.52).

The selective pathwise gradient estimator (SPGE) [START_REF] Schulman | Gradient Estimation Using Stochastic Computation Graphs[END_REF]] combines the two previous gradient estimators to alleviate their limitations: one has a large approximation error, and the other imposes a strong requirement for unbiasedness. The SPGE requires an additional program 𝑐 ′′ 𝑔 that is a reparameterisation of the guide 𝑐 𝑔 but needs not be 𝜃 -independent (unlike the PGE). An instance of 𝑐 ′′ 𝑔 for our example is given in Fig. 2, which changes the sample command for 𝑧 1 in 𝑐 𝑔 but keeps the one for 𝑧 2 . Note that the changed sample command for 𝑧 1 in 𝑐 ′′ 𝑔 uses a 𝜃 -independent probability distribution. Typically, 𝑐 ′′ 𝑔 is obtained by selecting a subset of the random variables in 𝑐 𝑔 and changing the sample commands for the selected variables such that their probability distributions become 𝜃 -independent; the sample commands for the unselected remain as they are. Given 𝑐 ′′ 𝑔 , the SPGE estimates ∇ 𝜃 L (𝜃 ) by following the recipe in Eq. ( 1) with 𝑞 ′′ 𝜃 (𝑧 1 , 𝑧 2 ) = 𝑝 𝑐 ′′ 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ) and

𝑔 ′′ 𝜃 (𝑧 1 , 𝑧 2 ) = ∇ 𝜃 𝑓 𝜃 (𝑧 ′′ 1 , 𝑧 ′′ 2 ) + 𝑓 𝜃 (𝑧 ′′ 1 , 𝑧 ′′ 2 ) • ∇ 𝜃 log 𝑞 ′′ 𝜃 (𝑧 1 , 𝑧 2 ) for (𝑧 ′′ 1 , 𝑧 ′′ 2 ) = 𝑣 𝑐 ′′ 𝑔 ,𝜃 (𝑧 1 , 𝑧 2 ).
(2)

Note that the estimation function 𝑔 ′′ Density semantics of the PPL. 6 We use a semantics of our language where commands are interpreted as calculators for densities, which may be unnormalised. Commands transform states, but in so doing, they compute densities of sampled random variables. More precisely, in the semantics, a command starts with an initial state that fixes not just the values of program variables but also those of all the random variables that are to be sampled during execution. When the command runs, it calculates the densities of those random variables at their given initial values, and also computes the probability density of all the observations, called likelihood. The product of all these densities and the likelihood becomes the so called unnormalised posterior density.

Let N be the set of natural numbers. Fix 𝑁 ∈ N with 𝑁 ≥ 1. Formally, the semantics uses the states of the following form:

𝜇 ∈ Name ≜ {(𝛼, 𝑖) | 𝛼 ∈ Str, 𝑖 ∈ N ∩ [0, 𝑁 )}, 𝑎 ∈ AVar ≜ {like} ∪ {pr 𝜇 , val 𝜇 , cnt 𝜇 | 𝜇 ∈ Name}, 𝑢, 𝑣 ∈ Var ≜ Name ⊎ PVar ⊎ AVar, 𝜎 ∈ St ≜ [Var → R], St[𝐾] ≜ [𝐾 → R] for 𝐾 ⊆ Var.
Here 𝜎 (𝜇) for 𝜇 ∈ Name is the initial value of the random variable 𝜇, which is used by the sample command and does not change during execution. For technical simplicity, the set Name has the restriction that the integer part of a name must be in [0, 𝑁 ). 7 The set AVar consists of four types of auxiliary variables. The auxiliary variable like stores the likelihood (i.e., the probability density of all the observations), and its value is initialised to 1 and changes whenever the observe command obs(𝑑, 𝑟 ) runs; the new value becomes the old times the density of the probability distribution 𝑑 at 𝑟 . The other auxiliary variables pr 𝜇 , val 𝜇 , and cnt 𝜇 are associated with a random variable 𝜇, standing for the "prior", "value", and "counter" of 𝜇. They are initialised with N (𝜎 (𝜇); 0, 1) (i.e., the density of the standard normal distribution at 𝜎 (𝜇)), 𝜎 (𝜇), and 0, respectively, and get updated by the sample command 𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒) where𝑛 denotes 𝜇. The command increases cnt 𝜇 by 1, so as to record the occurrence of a sampling event for 𝜇. Then, it looks up the given value 𝜎 (𝜇) of the random variable 𝜇, transforms the value to 𝑒 [𝜎 (𝜇)/𝑦], and stores the result in 𝑥 and val 𝜇 . Finally, the command computes the density of the distribution𝑑 at the looked-up value 𝜎 (𝜇), and updates pr 𝜇 with this density. The unnormalised posterior density (i.e., the joint density of all the random variables and observations) is then obtained by multiplying at the end of program execution the values of like and pr 𝜇 for all 𝜇 ∈ Name.

The formal semantics of expressions is standard, and has the following types:

𝑒 : St → R, 𝑏 : St → B, 𝑛 : St → Name, 𝑑 : St → D.
Here B is the set of booleans, i.e., true and false, and D is the set of positive probability-density functions on R, i.e., a subset of [R → (0, ∞)] whose elements are measurable functions that integrate to 1. The semantics is defined for a minor extension of the set of expressions where non-program variables are allowed to appear, such as (𝜇 + 𝑥). The interpretation of expressions is mostly standard. We semantics (e.g., [START_REF] Staton | Semantics for probabilistic programming: higher-order functions, continuous distributions, and soft constraints[END_REF]). Although the density semantics and the sampling semantics have different presentations, they are closely related and equivalent in a formal sense (see, e.g., [START_REF] Lee | Towards verified stochastic variational inference for probabilistic programs[END_REF]). We use the density semantics instead of the sampling semantics, because the gradient estimator ( §4) of our interest performs computation on (unnormalised) densities and it is easier for a program analysis ( §5) to work with the density semantics than the sampling semantics. 7 This restriction is often respected by probabilistic programs in practice, since they commonly sample random variables whose number is uniformly bounded over all traces; note, however, that it is not always respected (e.g., by programs from Bayesian nonparametrics). The uniform bound 𝑁 can often be found by a simple static analysis. This restriction along with the finiteness of show only the case for the name expressions 𝑛 ≡ name(𝛼, 𝑒): name(𝛼, 𝑒) 𝜎 ≜ create_name(𝛼, 𝑒 𝜎), where create_name : Str × R → Name is an operator converting a string-real pair to a name, defined by create_name(𝛼, 𝑟 ) = (𝛼, min{max{⌊𝑟 ⌋, 0}, 𝑁 -1}).8 Note that the types of the semantics of expressions imply that the evaluation of an expression always produces a value of the right type. In particular, it never generates an error. For instance, when an argument of an operator op or a distribution constructor dist N is outside its intended domain as in log(-3) and dist N (0, -2), or when the integer part of a name expression is outside [0, 𝑁 ) as in name("z", -1), our semantics does not generate an error. Instead, it returns some pre-chosen default value of the right type. This slightly unusual way of handling errors is also adopted in our semantics of commands to be presented shortly, and it lets us avoid the complexity caused by error handling when we formalise variational inference and develop our program analysis for smoothness properties.

The formal semantics of commands is also mostly standard with the handling of errors via default values, although its interpretation of sample and observe commands deserves special attention. Let ⊥ be an element not in St, and define St ⊥ to be the usual lifting of St with ⊥. That is, St ⊥ is a partially-ordered set St ⊎ {⊥} with the following order: for 𝜉, 𝜉 ′ ∈ St ⊥ , we have 𝜉 ⊑ 𝜉 ′ if and only if 𝜉 = ⊥ or 𝜉 = 𝜉 ′ . We write the standard lifting of a function 𝑓 :

St → St ⊥ by 𝑓 † : St ⊥ → St ⊥ (i.e., 𝑓 † (𝜉) ≜ if (𝜉 = ⊥) then 𝜉 else 𝑓 (𝜉)).
The semantics of a command 𝑐 is a map 𝑐 : St → St ⊥ , and is defined inductively as shown below: The interpretation uses the least fixed-point operator fix for continuous maps 𝐹 on the function space [St → St ⊥ ], where the function space is ordered pointwise and continuity means the one with respect to this order. It also uses the notation 𝑒 ′ [𝑒 ′′ /𝑦] to denote the substitution of 𝑦 in 𝑒 ′ by 𝑒 ′′ . According to this interpretation, 𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒 ′ ) increments the cnt 𝜇 variable for the name 𝑛 = 𝜇 so that the variable, which has 0 initially, records the number of times that the random variable with the same name 𝑛 is sampled during execution.

skip 𝜎 ≜ 𝜎, 𝑥 := 𝑒 𝜎 ≜ 𝜎 [𝑥 ↦ → 𝑒 𝜎], 𝑐; 𝑐 ′ 𝜎 ≜ 𝑐 ′ † ( 𝑐 𝜎), if 𝑏 {𝑐} else {𝑐 ′ } 𝜎 ≜ if ( 𝑏 𝜎 = true)
Having some cnt 𝜇 variable increased by 2 or larger at some point of execution is not an intended behaviour of a command 𝑐. That is, if 𝑐 is a well-designed command, every random variable with a fixed name should be sampled at most once during the execution of 𝑐. This intended behaviour of commands plays an important role in our results, and we refer to it using the following terminology.

St: 𝜎 ≜ [𝑥 ↦ → 0, 𝑦 ↦ → 0, ("a", 0) ↦ → 2, ("b", 0) ↦ → 4, cnt ("a",0) ↦ → 0, cnt ("b",0) ↦ → 0, • • • ],
where 𝑥, 𝑦 ∈ PVar denote program variables and ("a", 0), ("b", 0) ∈ Name denote random variables. Note that 𝜎 consists of three parts: the PVar part says that the values of 𝑥 and 𝑦 are both 0; the Name part says that the values of ("a", 0) and ("b", 0) are 2 and 4; and the AVar part says that ("a", 0) and ("b", 0) have not been sampled yet.

Next, consider a command𝑐 ≡ 𝑥 := sam(name("a", 0), dist N (-3, 1), 𝜆𝑧.𝑧); 𝑦 := sam(name("b", 0), dist N (5, 1), 𝜆𝑧.𝑧) . Given the command 𝑐 and the input state 𝜎, our density semantics computes the following output state 𝜎 ′ ∈ St: 𝜎 ′ ≜ 𝑐 𝜎 = [𝑥 ↦ → 2, 𝑦 ↦ → 4, ("a", 0) ↦ → 2, ("b", 0) ↦ → 4, cnt ("a",0) ↦ → 1, cnt ("b",0) ↦ → 1, pr ("a",0) ↦ → N (2; -3, 1), pr ("b",0) ↦ → N (4; 5, 1), • • • ]. The input/output states 𝜎 and 𝜎 ′ illustrate two aspects of our semantics. First, the semantics uses the Name part of an input state to determine the sampled values of sample commands: 𝑥 and 𝑦 in 𝜎 ′ take the values of 𝜎 (("a", 0)) = 2 and 𝜎 (("b", 0)) = 4. Second, the semantics records, in the AVar part of an output state, the densities of sample commands: pr ("a",0) and pr ("b",0) in 𝜎 ′ store the densities of the two sample commands in 𝑐 evaluated at 2 and 4. □

Variational inference. We consider the most common form of variational inference for Pyro-like PPLs where we are asked to learn a good approximation of the posterior of a given model, i.e., the conditional distribution of the model given a dataset. Typically, a parameterised approximate posterior is given in variational inference, and learning corresponds to finding good values of those parameters. A popular approach is to measure the quality of parameter values by the so called evidence lower bound (ELBO), and to optimise ELBO.

To translate what we have described so far to our context, we need to explain a general recipe for generating a density 𝑝 𝑐 for a command 𝑐, which is in general unnormalised (i.e., does not integrate to 1). The recipe specifies 𝑝 𝑐 as follows: for each

𝜎 𝜃 ∈ St[𝜃 ] with 𝜃 ⊆ PVar, 𝑝 𝑐,𝜎 𝜃 : St[Name] → [0, ∞) is defined by 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ) ≜ 𝑐 𝜎 (like) • 𝜇 ∈Name 𝑐 𝜎 (pr 𝜇 ) if 𝑐 𝜎 ∈ St and 𝑐 𝜎 (cnt 𝜇 ) ≤ 1 for all 𝜇 0 otherwise (3)
where 𝜎 = 𝜎 0 ⊕ 𝜎 𝜃 ⊕ 𝜎 𝑛 ∈ St, and the ⊕ operator combines two real-valued maps with disjoint domains in the standard way. Also, 𝜎 0 ∈ St[(PVar\𝜃 ) ⊎AVar] maps like to 1, and pr 𝜇 to N (𝜎 𝑛 (𝜇); 0, 1) and val 𝜇 to 𝜎 𝑛 (𝜇) for every 𝜇 ∈ Name, and all other variables to 0. Here St[Name] is understood as a measurable space constructed by taking the product of the |Name| copies of the measurable space R and the integral is taken over the uniform measure on St[Name] (i.e., the product of the |Name| copies of the Lebesgue measure on R).

In variational inference in our PPL context, we are given two commands 𝑐 𝑚 and 𝑐 𝑔 , called model and guide. We assume that (i) these commands always terminate and do not have a double-sampling error, (ii) some variables 𝜃 = {𝜃 1 , . . . , 𝜃 𝑘 } ⊆ PVar that only appear in 𝑐 𝑔 , not in 𝑐 𝑚 , are identified as parameters to be optimised, and (iii) the density 𝑝 𝑐 𝑔 ,𝜎 𝜃 of the guide 𝑐 𝑔 integrates to 1 and defines a probability distribution.9 Given the model-guide pair (𝑐 𝑚 , 𝑐 𝑔 ), a popular approach for variational inference is to solve the following optimisation problem approximately, argmax

𝜎 𝜃 E 𝑝 𝑐𝑔,𝜎 𝜃 (𝜎 𝑛 ) log(𝑝 𝑐 𝑚 (𝜎 𝑛 )/𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 𝑛 )) , (4)
when the expectation is well-defined for all 𝜎 𝜃 . The objective of this optimisation is the ELBO that we mentioned earlier. Here 𝑝 𝑐 𝑚 means 𝑝 𝑐 𝑚 ,𝜎 ′ 𝜃 for some/any 𝜎 ′ 𝜃 ; the choice of 𝜎 ′ 𝜃 does not matter since 𝑐 𝑚 does not access the parameters 𝜃 and so

𝑝 𝑐 𝑚 ,𝜎 ′ 𝜃 = 𝑝 𝑐 𝑚 ,𝜎 ′′ 𝜃 for all 𝜎 ′ 𝜃 , 𝜎 ′′ 𝜃 ∈ St[𝜃 ].
Often variational inference is applied when the model 𝑐 𝑚 is parameterised as well. In those cases, it asks for finding good parameters of the model 𝑐 𝑚 as well as those of the guide 𝑐 𝑔 . So, an algorithm for variational inference this time simultaneously learns a good model for given observations and a good approximate posterior for the learnt model. This more general form of variational inference can be easily accommodated in our setup. We just need to drop the condition that the parameters may not appear in 𝑐 𝑚 , and to use 𝑝 𝑐 𝑚 ,𝜎 𝜃 instead of 𝑝 𝑐 𝑚 in the optimisation objective in Eq. ( 4): argmax

𝜎 𝜃 L (𝜎 𝜃 ) for L (𝜎 𝜃 ) ≜ E 𝑝 𝑐𝑔,𝜎 𝜃 (𝜎 𝑛 ) log(𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 𝑛 )/𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 𝑛 )) .
(5)

The rest of the paper focuses on this general form of variational inference (often called model learning).

SELECTIVE PATHWISE GRADIENT ESTIMATOR

We consider a gradient-based algorithm for the optimisation problem in Eq. ( 5). The algorithm finds a good 𝜎 𝜃 by repeatedly estimating the gradient of the optimisation objective at the current 𝜎 𝜃 ,

grad_est(𝜎 𝜃 ) ≈ ∇ 𝜃 E 𝑝 𝑐𝑔,𝜎 𝜃 (𝜎 𝑛 ) log(𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 𝑛 )/𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 𝑛 )) ,
and updating 𝜎 𝜃 with the estimate under a learning rate 𝜂 > 0, that is,

𝜎 𝜃 ← 𝜎 𝜃 + 𝜂 • grad_est(𝜎 𝜃 ).
Note that the core of the algorithm lies in the computation of grad_est(𝜎 𝜃 ).

In this section, we describe a particular algorithm for the gradient computation, called selective pathwise gradient estimator (SPGE), which is often regarded as the algorithm of choice and corresponds to the inference algorithm developed for stochastic computation graphs [START_REF] Schulman | Gradient Estimation Using Stochastic Computation Graphs[END_REF] and implemented for Pyro. Our description of the SPGE takes the often-ignored aspect of customising the SPGE algorithm for PPLs seriously, and it is accompanied with a novel formal analysis of the customisation ( §4.1 and §4.2). Our analysis clearly identifies information about probabilistic programs that is useful for this customised SPGE algorithm, and prepares the stage for our program analysis for smoothness properties in §5 ( §4.2 and §4.3).

Program Transformation

We start by describing a program transformation that changes some sample commands in a given probabilistic program. This transformation is used crucially by the SPGE.

The key component of the transformation is a partial function 𝜋 called reparameterisation plan, which has the type NameEx × DistEx × LamEx ⇀ DistEx × LamEx. Here NameEx, DistEx, and LamEx denote the sets of name expressions, distribution expressions, and lambda expressions of the form 𝜆𝑦.𝑒, respectively. The plan 𝜋 specifies how we transform sample commands. Concretely, assume that we are given 𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒). We check whether 𝜋 (𝑛, 𝑑, 𝜆𝑦.𝑒) is defined or not. If not, we keep the original sample command. Otherwise, if 𝜋 (𝑛, 𝑑, 𝜆𝑦.𝑒) is (𝑑 ′ , 𝜆𝑦 ′ .𝑒 ′ ), we replace the command with 𝑥 := sam(𝑛, 𝑑 ′ , 𝜆𝑦 ′ .𝑒 ′ ).

A natural extension of this intended transformation of 𝜋 leads to the following program transformation for a general command 𝑐, denoted by 𝑐 𝜋 :

skip 𝜋 ≜ skip, 𝑥 := 𝑒 𝜋 ≜ 𝑥 := 𝑒, 𝑐; 𝑐 ′ 𝜋 ≜ 𝑐 𝜋 ; 𝑐 ′ 𝜋 , if 𝑏 {𝑐} else {𝑐 ′ } 𝜋 ≜ if 𝑏 {𝑐 𝜋 } else {𝑐 ′ 𝜋 }, while 𝑏 {𝑐} 𝜋 ≜ while 𝑏 {𝑐 𝜋 }, 𝑥 := sam(𝑛, 𝑑, 𝑙) 𝜋 ≜ 𝑥 := sam(𝑛, 𝑑 ′ , 𝑙 ′ ) if ∃(𝑑 ′ , 𝑙 ′ ). 𝜋 (𝑛, 𝑑, 𝑙) = (𝑑 ′ , 𝑙 ′ ) 𝑥 := sam(𝑛, 𝑑, 𝑙) otherwise, obs(𝑑, 𝑟 ) 𝜋 ≜ obs(𝑑, 𝑟 ).
The transformation recursively traverses 𝑐, and applies 𝜋 to all the sample commands in 𝑐. Note that for any 𝜋, there exists a total function 𝜋 ′ such that 𝑐 𝜋 = 𝑐 𝜋 ′ for all 𝑐; the 𝜋 ′ coincides with 𝜋 in the domain of 𝜋, and outside of this domain, it is the identity function. But such 𝜋 ′ loses information about the domain of 𝜋, which plays a crucial role in our formalisation of the SPGE.

We are primarily interested in semantics-preserving instances of • 𝜋 . The next definition helps us to identify such instances. Definition 4.1. A reparameterisation plan 𝜋 is valid if for all 𝑛 ∈ NameEx, 𝑑, 𝑑 ′ ∈ DistEx, and (𝜆𝑦.𝑒), (𝜆𝑦 ′ .𝑒 ′ ) ∈ LamEx such that 𝜋 (𝑛, 𝑑, 𝜆𝑦.𝑒) = (𝑑 ′ , 𝜆𝑦 ′ .𝑒 ′ ), the following condition holds: for all states 𝜎 ∈ St and measurable subsets

𝐴 ⊆ R, ∫ 1 [ 𝑒 [𝑟 /𝑦 ] 𝜎 ∈𝐴] • 𝑑 𝜎 (𝑟 ) 𝑑𝑟 = ∫ 1 [ 𝑒 ′ [𝑟 /𝑦 ′ ] 𝜎 ∈𝐴] • 𝑑 ′ 𝜎 (𝑟 ) 𝑑𝑟 . □ (6)
The condition says that the distribution obtained by sampling from 𝑑 and applying 𝜆𝑦.𝑒 is the same as that obtained by sampling from 𝑑 ′ and applying 𝜆𝑦 ′ .𝑒 ′ . An example of a widely-used valid reparameterisation plan maps its input as follows, whenever defined:

𝜋 0 (𝑛, dist N (𝑒 1 , 𝑒 2 ), 𝜆𝑦.𝑒 3 ) = (dist N (0, 1), 𝜆𝑦.𝑒 3 [(𝑦× √ 𝑒 2 +𝑒 1 )/𝑦])
, where we assume𝑦 does not appear in𝑒 1 and𝑒 2 , the substitution in 𝜋 0 expresses the composition of two functions 𝜆𝑦.𝑒 3 and 𝜆𝑦.(𝑦 × √ 𝑒 2 +𝑒 1 ), and √denotes a squareroot operator that handles non-positive arguments in the same way as dist N (𝑒, -)

does: if 𝑒 2 𝜎 ≤ 0 and dist N (𝑒 1 , 𝑒 2 ) 𝜎 = 𝜆𝑟 . N (𝑟 ; 𝑒 1 𝜎, 𝑟 2 ) for some 𝑟 2 > 0, then √ 𝑒 2 𝜎 = √ 𝑟 2 .
The above plan satisfies the condition in Eq. ( 6), because𝑦 × √ 𝑟 2 +𝑟 1 with a sample𝑦 from N (0, 1) is distributed by N (𝑟 1 , 𝑟 2 ). We now show that • 𝜋 with a valid 𝜋 preserves semantics. For a command 𝑐 and

𝜎 𝜃 ∈ St[𝜃 ], define the value function 𝑣 𝑐,𝜎 𝜃 : St[Name] → St[Name] as follows: 𝑣 𝑐,𝜎 𝜃 (𝜎 𝑛 ) (𝜇) ≜ let 𝜎 ≜ 𝜎 0 ⊕ 𝜎 𝜃 ⊕ 𝜎 𝑛 in 𝑐 𝜎 (val 𝜇 ) if 𝑐 𝜎 ∈ St and 𝑐 𝜎 (cnt 𝜇 ′ ) ≤ 1 for all 𝜇 ′ 0 otherwise
where 𝜎 0 ∈ St[(PVar \ 𝜃 ) ⊎ AVar] maps like to 1, and pr 𝜇 to N (𝜎 𝑛 (𝜇); 0, 1) and val 𝜇 to 𝜎 𝑛 (𝜇) for every 𝜇 ∈ Name, and it also maps all the other variables to 0. The value function basically applies the lambda functions in 𝑐's sample commands to the corresponding random variables. The next theorem proves that if 𝜋 is valid, the program transformation • 𝜋 preserves the semantics in the sense that the integral of a function ℎ remains the same under 𝑐 and 𝑐 𝜋 for any 𝑐. Note that the two integrals in the theorem are connected via the value functions of 𝑐 and 𝑐 𝜋 .

Theorem 4.2. Let 𝜋 be a valid reparameterisation plan, and 𝑐 be a command. Then, for all 𝜎 𝜃 ∈ St[𝜃 ] and all measurable ℎ : St[Name] → R, we have

∫ 𝑑𝜎 𝑛 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ) • ℎ(𝑣 𝑐,𝜎 𝜃 (𝜎 𝑛 )) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • ℎ(𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ))
where the left integral is defined if and only if the right integral is defined.

Remark 4.3. One immediate yet important consequence of the theorem is that if 𝑝 𝑐,𝜎 𝜃 is a probability density, so is 𝑝 𝑐 𝜋 ,𝜎 𝜃 . This consequence will be used in §4.2 and the proof of Theorem 4.5 later. □

Gradient Estimator via Program Transformation

Let 𝑐 be a command that always terminates and does not have a double-sampling error, and let

𝜎 𝜃 ∈ St[𝜃 ].
We define the partial density function 𝑝 ⟨𝑆 ⟩ 𝑐,𝜎 𝜃 of 𝑐 over a subset 𝑆 ⊆ Name as

𝑝 ⟨𝑆 ⟩ 𝑐,𝜎 𝜃 : St[Name] → (0, ∞), 𝑝 ⟨𝑆 ⟩ 𝑐,𝜎 𝜃 (𝜎 𝑛 ) ≜ 𝜇 ∈𝑆 𝑐 (𝜎 0 ⊕ 𝜎 𝜃 ⊕ 𝜎 𝑛 ) (pr 𝜇 ),
where 𝜎 0 is set as in the definition of 𝑝 𝑐,𝜎 𝜃 in Eq. ( 3). The partial density 𝑝 ⟨𝑆 ⟩ 𝑐,𝜎 𝜃 is essentially the full density 𝑝 𝑐,𝜎 𝜃 in Eq. ( 3) with the omission of the factors not mentioned in 𝑆. Intuitively, it computes the density of the random variables in 𝑆 conditioned on the random variables outside of 𝑆.

The SPGE computes an approximate gradient of the objective L in Eq. ( 5) using the program transformation in the previous subsection. Its inputs are a model 𝑐 𝑚 , a guide 𝑐 𝑔 , parameters 𝜃 to optimise, and a reparameterisation plan 𝜋, where

• 𝑐 𝑚 , 𝑐 𝑔 , and 𝑐 𝑔 𝜋 always terminate and do not have a double-sampling error, and • 𝑐 𝑔 defines the normalised probability density 𝑝 𝑐 𝑔 ,𝜎 𝜃 for all

𝜎 𝜃 ∈ St[𝜃 ]. (7)
Given these inputs, the SPGE computes an approximate gradient in three steps. First, it defines the set rv(𝜋) ⊆ Name of random variables to be reparameterised:

rv(𝜋) ≜ {(𝛼, 𝑖) ∈ Name | (name(𝛼, _), _, _) ∈ dom(𝜋)}
where _ means some existentially quantified (meta) variable. Second, the SPGE transforms the guide 𝑐 𝑔 to 𝑐 𝑔 𝜋 , and draws a sample σ𝑛 from 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 . 10 Drawing a sample σ𝑛 makes sense here since 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 is a probability density (i.e., it normalises to 1) by Remark 4.3. Another important point is that drawing σ𝑛 can be done simply by executing 𝑐 𝑔 𝜋 in the standard sampling semantics (not in our density semantics), where each sample command is interpreted as a random draw, not as a density calculator. Third, the SPGE computes the following approximation of ∇ 𝜃 L (𝜎 𝜃 ) and returns it as a result:

grad_est(𝜎 𝜃 ; σ𝑛 ) ≜ ∇ 𝜃 log 𝑝 ⟨Name\rv (𝜋 ) ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) • log(𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 )/𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 )) -∇ 𝜃 log 𝑝 ⟨rv (𝜋 ) ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) + ∇ 𝜃 log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ), for 𝜎 ′ 𝑛 ≜ 𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 ( σ𝑛 ). (8)
Recall that if a command 𝑐 always terminates, both the partial density 𝑝 ⟨𝑆 ⟩ 𝑐,𝜎 𝜃 (𝜎 𝑛 ) and the full density 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ) can be computed simply by executing 𝑐 in our semantics and calculating the defining formulas of both densities from the final state of the execution. Thus, all the terms in grad_est can be computed by executing 𝑐 𝑔 and 𝑐 𝑚 according to our density semantics or differentiating the results of these executions via, for instance, automatic differentiation as done in Pyro. Note that grad_est applies two non-trivial optimisations, when compared with the (naive) SPGE explained in Eq. (2): its first term involves a partial density of 𝑐 𝑔 instead of the full density of 𝑐 𝑔 𝜋 , and its second term involves (again) a partial density of 𝑐 𝑔 instead of the full density of 𝑐 𝑔 .

Is the SPGE correct in any sense? The answer depends on its inputs. If the inputs satisfy the requirements that we will explain soon, the result of the SPGE is precisely ∇ 𝜃 L (𝜎 𝜃 ) on average, that is, ∇ 𝜃 L (𝜎 𝜃 ) = E[grad_est(𝜎 𝜃 ; σ𝑛 )], where the expectation is taken over the sample σ𝑛 used by the SPGE. This property is called unbiasedness, and it plays the crucial role for ensuring that parameters updated iteratively with estimated gradients converge to a local optimum.

Let us now spell out the requirements on the inputs of the SPGE. To do so, we need to introduce one further concept for the reparameterisation plans 𝜋. Definition 4.4. A reparameterisation plan 𝜋 is simple if for all (𝑛, 𝑑, 𝜆𝑦.𝑒) and (𝑛 ′ , 𝑑 ′ , 𝜆𝑦 ′ .𝑒 ′ ) in NameEx × DistEx × LamEx such that 𝑛 and 𝑛 ′ have the same string part, we have (𝑛, 𝑑, 𝜆𝑦.𝑒) ∈ dom(𝜋) ⇐⇒ (𝑛 ′ , 𝑑 ′ , 𝜆𝑦 ′ .𝑒 ′ ) ∈ dom(𝜋).

□

The simplicity is one of the requirements that the SPGE imposes on 𝜋. It ensures the following property of the set rv(𝜋), which the SPGE relies on when computing grad_est by Eq. ( 8): rv(𝜋) (and Name \ rv(𝜋)) over-approximates the set of random variables that, if sampled, are (and are not) reparameterised by • 𝜋 . Specifically, it forbids 𝜋 from using any syntax-specific information of the arguments of a sample command when it decides whether to transform the command or not. All the requirements of the SPGE, including the simplicity just explained, are summarised in the next theorem.

Theorem 4.5. Let 𝑐 𝑚 , 𝑐 𝑔 , and 𝜋 be the inputs to the SPGE (i.e., they satisfy the assumptions in Eq. ( 7)). Suppose that L (𝜎 𝜃 ) and ∇ 𝜃 L (𝜎 𝜃 ) are well-defined for every 𝜎 𝜃 ∈ St[𝜃 ]. Further, assume that every

∇ 𝜃 ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) = ∫ 𝑑𝜎 𝑛 ∇ 𝜃 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) , ∇ 𝜃 ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 ∇ 𝜃 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 )
.

In the second equation, we write 𝜎 ′ 𝑛 for 𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ). To be clear, 𝑓 :

St[𝐾] → R 𝑛 for 𝐾 ⊆ Var is said to be differentiable in 𝐾 ′ ⊆ 𝐾 jointly if for any 𝜏 ∈ St[𝐾 \ 𝐾 ′ ], 𝑓 | [𝜏 ] : St[𝐾 ′ ] → R 𝑛 is (jointly) differentiable, where 𝑓 | [𝜏 ] (𝜎) ≜ 𝑓 (𝜎 ⊕ 𝜏).
In this work, we focus on the requirements (R2) and (R3) about smoothness. They require that five density functions of 𝑐 𝑚 , 𝑐 𝑔 , and 𝑐 𝑔 𝜋 , and the value function of 𝑐 𝑔 𝜋 be differentiable in certain variables. We will develop a program-analysis framework to check these differentiability requirements soundly and automatically ( §5), and will describe an algorithm to the SPGE variable-selection problem, using the developed analysis framework ( §6). The remaining requirements (R1), (R4), and (R5) are of less interest in this work. (R1) and (R4) can be guaranteed by simple syntactic checks and our way of constructing 𝜋 (Lemma D.1). (R5) follows from (R2) and (R3) under a regularity condition on densities (Theorem D.2) which is usually satisfied in practice; in some cases, however, the condition might not hold, and we leave it as future work to automatically discharge the condition (which is about the integrability of local Lipschitz constants of densities) or more generally (R5). 11 We point out that Pyro uses the SPGE in their inference engine, but without checking the above requirements. In particular, its default option simply uses the 𝜋 that transforms all the continuous random variables in a guide, and this can easily violate the requirements and make the SPGE biased.

Local Lipschitzness for Relaxed Requirements

In Theorem 4.5, we considered the requirements (R2) and (R3) about the differentiability of density and value functions, as a sufficient condition for the unbiasedness of the SPGE. They are, however, sometimes too strong to hold in practice due to the use of popular non-differentiable functions (e.g., ReLU). As we will see in §7, the requirements are indeed violated by some representative Pyro programs even though the conclusion of Theorem 4.5 holds for those programs (i.e., the estimated gradients by the SPGE for those programs are unbiased).

To validate the unbiasedness of the SPGE for more examples in practice, we consider the following relaxation of the requirements (R2) and (R3), which changes differentiability to local Lipschitzness:

(R2') The functions in (R2) are locally Lipschitz in 𝜃 ∪ rv(𝜋) jointly. (R3') For every 𝜎 𝑛 ∈ St[Name], the functions in (R3) are locally Lipschitz in 𝜃 jointly.

11 Some sufficient conditions for a regularity condition similar to the one considered here have been studied for certain classes of model-guide pairs, e.g., in [START_REF] Lee | Towards verified stochastic variational inference for probabilistic programs[END_REF] ') and (R3') amounts to relaxing the requirements of (R2) and (R3) in practice. We choose local Lipschitzness as an alternative to differentiability in (R2) and (R3) for two main reasons. First, local Lipschitzness is satisfied by most functions used in practice, which can even be non-differentiable (e.g., ReLU). This would allow us to validate the unbiasedness of the SPGE for more programs, even when they use non-differentiable functions. Second, using local Lipschitzness in (R2) and (R3) does not break the results given in §4.2, even though local Lipschitzness is more practically permissive than differentiability (as explained above). In particular, we have a counterpart of Theorem 4.5 that uses (R2') and (R3') instead of (R2) and (R3): Theorem 4.6. Consider the setup of Theorem 4.5. Then, for all 𝜎 𝜃 ∈ St[𝜃 ], Eq. (9) holds if 𝜋 satisfies the requirements (R1), (R2'), (R3'), (R4), and (R5).

We can obtain Theorem 4.6 thanks to three properties of local Lipschitzness: locally Lipschitz functions are closed under function composition, have well-defined gradients almost everywhere, and satisfy the chain rule almost everywhere in restricted settings (Lemma E.1). Although the latter two properties are weaker than the corresponding properties of differentiability (i.e., differentiable functions always have well-definedness gradients and satisfy the chain rule), they are still strong enough to prove the theorem.

As in §4.2, we focus on the requirements (R2') and (R3') and less on (R1), (R4), and (R5). Note that (R1) and (R4) can be checked syntactically in the same way as discussed in §4.2. On the other hand, (R5) follows from (R2') and (R3') now under two (instead of one) regularity conditions on densities (Theorem E.2), where the first condition is the one mentioned in §4.2 and the second (new) condition is about some form of almost-everywhere differentiability of densities. Although we believe that the two regularity conditions would usually hold in practice, they can be violated in some cases that involve locally Lipschitz, non-differentiable functions; hence, it would be worthwhile to devise an automatic way of checking the two conditions or more generally (R5), which we leave as future work.

Because local Lipschitzness property has wider coverage than differentiability in practice while ensuring that the results in §4.2 remain valid, our implementation and experiments consider the option of using (R2') and (R3') as well as that of using (R2) and (R3); see §5.3- §7 for details.

PROGRAM ANALYSIS FOR SMOOTHNESS

Recall that our goal is to develop an algorithm for the SPGE variable-selection problem in Definition 2.1, which asks for finding a large set 𝑆 of random variables with a certain property when given a model 𝑐 𝑚 , a guide 𝑐 𝑔 , and a reparameterisation plan 𝜋 0 . When rephrased using the terminologies that we covered so far, finding such an 𝑆 amounts to finding a restriction 𝜋 of the given 𝜋 0 such that (𝑐 𝑚 , 𝑐 𝑔 , 𝜋) satisfies the requirements in Theorem 4.5 (or 4.6). Thus, the key for developing a desired algorithm for the problem lies in constructing an automatic method for proving that the requirements in Theorem 4.5 (or 4.6), in particular, the smoothness requirements (R2) and (R3) (or (R2') and (R3')) are met. In this section, we propose a program analysis for smoothness properties, which can help find 𝜋 that meets the smoothness requirements, and which, together with the optimiser in the next section, leads to an algorithm for solving the SPGE variable-selection problem.

We first define a parametric abstraction for smoothness properties ( §5.1). We then describe a program analysis based on this abstraction and prove the soundness of the analysis ( §5.2). We finally instantiate the analysis to differentiability and local Lipschitzness ( §5.3). The results in this section are not limited to PPLs, but are applicable to general imperative programming languages.

Parametric Abstraction for Smoothness Properties

At a high level, our parametric abstraction for smoothness properties is built out of two components. The first is a predicate over commands that expresses a target smoothness property but in a conditional form. The predicate is parameterised by two sets of variables, 𝐾 for the input variables and 𝐿 for the output variables. Intuitively, the predicate holds for a command if conditioning the input variables outside of 𝐾 to any fixed values and varying only the ones in 𝐾 makes the command a smooth function on the output variables in 𝐿. Our program analysis tracks a conditional smoothness property formalised by this predicate, and in so doing, it identifies a smooth part of a given command, even when the command fails to be so with respect to some variables.

The second component is also a predicate over commands, but it deals with dependency, instead of smoothness. It is again parameterised by 𝐾 and 𝐿, and expresses that to compute the output variables in 𝐿, a command accesses only the input variables in 𝐾. Our program analysis tracks dependency formalised by this predicate, so as to achieve high precision, especially when handling sequential composition. To see this, imagine that we want to check the differentiability of a sequence 𝑐; 𝑐 ′ . A natural approach is to use the chain rule. If the dependency-tracking part of our analysis is missing, in order to establish that the output on a variable 𝑣 by the sequence is differentiable in an input variable 𝑢, the analysis should show the output on 𝑣 by the second command 𝑐 ′ is differentiable in all variables, and the output on any variable by the first command 𝑐 is differentiable in 𝑢. This requirement on 𝑐 and 𝑐 ′ is too strong. Often, 𝑐 ′ uses only a small number of variables to compute 𝑣, and it is sufficient to require that just on those used variables, 𝑐 should be differentiable in 𝑢. Similarly, 𝑐 commonly updates only a small number of variables using 𝑢, and it is enough to require that just in those 𝑢-dependent variables, the second command 𝑐 ′ is differentiable when computing the output 𝑣. The dependency-tracking part lets our analysis carry out such reasoning and achieve better precision. Formally, this means our analysis uses a version of reduced product [START_REF] Cousot | Systematic design of program analysis frameworks[END_REF] between dependency analysis and the analysis that tracks the target smoothness property.

We now formally describe each of these components as well as their combination.

5.

over [St → St ⊥ ].
The reason is that, as mentioned above, the analysis aims at a conditional variant of the traditional notion of smoothness. For instance, instead of checking that a function 𝑓 : St → St ⊥ is differentiable on St\𝑓 -1 ({⊥}), the analysis proves differentiability conditioned on certain variables being fixed: if we fix the input variables in Var\𝐾 and vary just those in 𝐾 in the initial state, and look at the output variables in 𝐿 only, then the function 𝑓 becomes differentiable, although it might not be so when all input/output variables are considered. To express this, we need the whole family of 𝜙 𝐾,𝐿 predicates. This notion of conditional differentiability is similar to, but not the same as, so called partial differentiability. Partial differentiability in 𝐾 says that, for every 𝑣 ∈ 𝐾, if we fix all the input variables except 𝑣, including those in 𝐾 \ {𝑣 }, and consider the output variables in 𝐿 only, 𝑓 becomes differentiable.12 As we will show in Remark 5.12, the set of partially-differentiable functions is not closed under a certain operator, but we need the closure to ensure that our program analysis is sound. Our conditional differentiability does not suffer from this issue. A direct but naive way of implementing this intuition is to let D ♯ be the collection of all the constraints of this form, but it permits too many constraints and leads to a costly program analysis. We take a more economical alternative that further restricts the allowed form of the constraints. The alternative requires that the conjunction from above should be constructed out of two mappings 𝑝 and 𝑑 from output variables to input variable sets, and a set 𝑉 of input variables. The 𝑝 component describes smoothness, and the 𝑑 and 𝑉 components dependency. They together encode the constraint

5

𝑣 ∈Var Φ(𝑓 , 𝑝 (𝑣), {𝑣 }) ∧ 𝑢 ∈Var Δ(𝑓 , 𝑑 (𝑢), {𝑢}) ∧ Δ(𝑓 , 𝑉 , ∅).
Thus, a function 𝑓 ∈ [St → St ⊥ ] satisfies the constraint encoded by 𝑝, 𝑑, and 𝑉 if (i) for every output variable 𝑣, when we fix the values of all the input variables outside of 𝑝 (𝑣), the (partial) function 𝜎 ↦ -→ 𝑓 (𝜎) (𝑣) is smooth (e.g., differentiable); (ii) for every output variable 𝑢, the (partial) function 𝜎 ↦ -→ 𝑓 (𝜎) (𝑢) does not access any variable outside of 𝑑 (𝑢) to compute the value of 𝑢; and (iii) the values of input variables in 𝑉 determine whether 𝑓 returns ⊥ or not. Example 5.5 (Differentiability). Consider the setup of Example 5.1 and the program 𝑐 ≡ (𝑦 := 𝑥 * 𝑥; if (𝑥 ≥ 0) {𝑠 := 1} else {𝑠 := -1}). Let (𝑝, 𝑑, 𝑉 ) be the smallest abstract state that describes the program. In this program, 𝑠 is not differentiable in 𝑥, but 𝑦 is. So, 𝑝 (𝑠) = Var \ {𝑥 } ⊇ {𝑦, 𝑠} and 𝑝 (𝑦) = Var ⊇ {𝑥, 𝑦, 𝑠}. Note that 𝑝 (𝑠) contains the input variables 𝑠 and 𝑦 because by not depending on those input variables, the output 𝑠 is differentiable in those variables. For the dependency part, we have 𝑑 (𝑠) = 𝑑 (𝑦) = {𝑥 } and 𝑉 = ∅. □

Parametric Static Program Analysis

Our program analysis is based on abstract interpretation [START_REF] Cousot | Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints[END_REF], and computes an approximation of the concrete semantics 𝑐 of a given command 𝑐 using the abstract domain D ♯ . We formalise this computation by the abstract semantics of 𝑐, which defines 𝑐 ♯ ∈ D ♯ by induction on the structure of commands, and over-approximates 𝑐 in the sense of the concretization 𝛾 in Eq. (10).

5.2.1 Analysis Definition. Fig. 3 shows the abstract semantics of 𝑐 ♯ . The overall structure of the semantics follows the standard compositional semantics of an imperative language. For instance, the abstract semantics of sequential composition is defined in terms of those of constituent commands, and the semantics of a loop is the least fixed point of a monotone operator over D ♯ . However, the specifics of the semantics include non-standard details, and we spell them out by going through the defining clauses of 𝑐 ♯ . The definition of skip ♯ formalises the effect of skip on smoothness and dependency. The definition says that skip computes each output variable 𝑣 in a smooth manner in all input variables, and in so doing, it creates the dependency between the variable 𝑣 to itself at the input state. The 𝑉 part of skip ♯ is the empty set since skip always terminates. The next case is 𝑥 := 𝑒. Its abstract semantics records the smoothness and dependency information of the updated variable 𝑥 by analysing the expression 𝑒. For the smoothness part, the semantics invokes the subroutine 𝑒 ♯ that computes an under-approximation of the set of variables in which the expression 𝑒 is smooth (and thus over-approximates the smoothness property of 𝑒). 13 For the dependency part, 𝑥 := 𝑒 ♯ computes the set of all the free variables of 𝑒 so as to get an over-approximation of all variables that may affect the value of 𝑒. For variables other than 𝑥, 𝑥 := 𝑒 ♯ behaves like skip ♯ . The abstract semantics of a sequence 𝑐; 𝑐 ′ composes those of the sub-commands 𝑐 and 𝑐 ′ . It uses the liftings 𝑓 ∪ , 𝑓 ∩ : P (Var) → P (Var) of functions 𝑓 of type Var → P (Var), which are defined as follows: 𝑓 ∪ (𝑉 ) ≜ 𝑣 ∈𝑉 𝑓 (𝑣) and 𝑓 ∩ (𝑉 ) ≜ 𝑣 ∈𝑉 𝑓 (𝑣). The abstract semantics 𝑐; 𝑐 ′ ♯ constructs 13 The subroutine 𝑒 ♯ ⊆ Var is defined inductively on 𝑒, and differs for different target smoothness properties. For instance, if the target property is differentiability, we have

𝑟 ♯ ≜ 𝑥 ♯ ≜ Var, 𝑒 1 + 𝑒 2 ♯ ≜ 𝑒 1 ♯ ∩ 𝑒 2 ♯ , ReLU(𝑒) ♯ ≜ Var∩fv (𝑒) 𝑐 , etc.
On the other hand, if the target property is local Lipschitzness, the subroutine changes for some cases: e.g., ReLU(𝑒) ♯ ≜ 𝑒 ♯ .

Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

skip ♯ ≜ (𝜆𝑣.Var, 𝜆𝑣 .{𝑣 }, ∅),

𝑥 := 𝑒 ♯ ≜ ((𝜆𝑣. 𝑣 ≡ 𝑥 ? 𝑒 ♯ : Var), (𝜆𝑣. 𝑣 ≡ 𝑥 ? fv(𝑒) : {𝑣 }), ∅), 𝑐; 𝑐 ′ ♯ ≜ let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯ and (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯ in (𝑝 ′′ , 𝑑 ′′ , 𝑉 ′′ ) where 𝑝 ′′ (𝑣) ≜ (𝑉 ∪ 𝑝 ∩ (𝑑 ′ (𝑣)) 𝑐 ∪ 𝑑 ∪ (𝑝 ′ (𝑣) 𝑐 )) 𝑐 , 𝑑 ′′ (𝑣) ≜ 𝑉 ∪ 𝑑 ∪ (𝑑 ′ (𝑣)), and 𝑉 ′′ ≜ 𝑉 ∪ 𝑑 ∪ (𝑉 ′ ), if 𝑏 {𝑐} else {𝑐 ′ } ♯ ≜ let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯ and (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯ in (𝑝 ′′ , 𝑑 ′′ , 𝑉 ′′ ) where 𝑝 ′′ (𝑣) ≜ fv(𝑏) 𝑐 ∩ 𝑝 (𝑣) ∩ 𝑝 ′ (𝑣), 𝑑 ′′ (𝑣) ≜ fv(𝑏) ∪ 𝑑 (𝑣) ∪ 𝑑 ′ (𝑣), and 𝑉 ′′ ≜ fv(𝑏) ∪ 𝑉 ∪ 𝑉 ′ , while 𝑏 {𝑐} ♯ ≜ let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯ in fix 𝐹 ♯ where 𝐹 ♯ (𝑝 0 , 𝑑 0 , 𝑉 0 ) ≜ (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ), 𝑝 ′ (𝑣) ≜ fv(𝑏) 𝑐 ∩ (𝑉 ∪ 𝑝 ∩ (𝑑 0 (𝑣)) 𝑐 ∪ 𝑑 ∪ (𝑝 0 (𝑣) 𝑐 )) 𝑐 , 𝑑 ′ (𝑣) ≜ fv(𝑏) ∪ (𝑉 ∪ 𝑑 ∪ (𝑑 0 (𝑣))) ∪ {𝑣 }, and 𝑉 ′ ≜ fv(𝑏) ∪ (𝑉 ∪ 𝑑 ∪ (𝑉 0 )), obs(dist N (𝑒 1 , 𝑒 2 ), 𝑟 ) ♯ ≜ (𝑝, 𝑑, ∅)
where

𝑝 (𝑣) ≜ (𝑣 ≡ like) ? like × pdf N (𝑟 ; 𝑒 1 , 𝑒 2 ) ♯ : Var, and 𝑑 (𝑣) ≜ (𝑣 ≡ like) ? {like} ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 ) : {𝑣 }, 𝑥 := sam(𝑛, dist N (𝑒 1 , 𝑒 2 ), 𝜆𝑦.𝑒 ′ ) ♯ ≜ (𝑝, 𝑑, ∅) for 𝑛 = name(𝛼, 𝑟 ) with 𝑟 ∈ R
where 𝜇 ≜ create_name(𝛼, 𝑟 ),

𝑝 (𝑣) ≜            𝑒 ′ [𝜇/𝑦] ♯ if 𝑣 ∈ {𝑥, val 𝜇 } pdf N (𝜇; 𝑒 1 , 𝑒 2 ) ♯ if 𝑣 ≡ pr 𝜇 cnt 𝜇 + 1 ♯ if 𝑣 ≡ cnt 𝜇 Var otherwise,
and

𝑑 (𝑣) ≜        fv(𝑒 ′ [𝜇/𝑦]) if 𝑣 ∈ {𝑥, val 𝜇 } {𝜇} ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 ) if 𝑣 ≡ pr 𝜇 {𝑣 } otherwise, 𝑥 := sam(𝑛, dist N (𝑒 1 , 𝑒 2 ), 𝜆𝑦.𝑒 ′ ) ♯ ≜ (𝑝, 𝑑, ∅) for 𝑛 = name(𝛼, 𝑒) with 𝑒 ∉ R where 𝑝 (𝑣) ≜                fv(𝑒) 𝑐 ∩ 𝜇=(𝛼,_) ∈Name 𝑒 ′ [𝜇/𝑦] ♯ if 𝑣 ≡ 𝑥 fv(𝑒) 𝑐 ∩ 𝑒 ′ [𝜇/𝑦] ♯ if 𝑣 ≡ val 𝜇 for 𝜇 = (𝛼, _) fv(𝑒) 𝑐 ∩ pdf N (𝜇; 𝑒 1 , 𝑒 2 ) ♯ if 𝑣 ≡ pr 𝜇 for 𝜇 = (𝛼, _) fv(𝑒) 𝑐 ∩ cnt 𝜇 + 1 ♯ if 𝑣 ≡ cnt 𝜇 for 𝜇 = (𝛼, _) Var otherwise,
and the dependency part 𝑑 ′′ by composing 𝑑 from 𝑐 ♯ and 𝑑 ′ from 𝑐 ′ ♯ after lifting the former. Note the inclusion of the set 𝑉 in the definition of 𝑑 ′′ . This is to account for the case that 𝑑 ′ (𝑣) in the definition is the empty set; in that case, 𝑑 ∪ (𝑑 ′ (𝑣)) is empty as well and does not have any information about termination. The smoothness part 𝑝 ′′ of 𝑐; 𝑐 ′ ♯ is more involved, and implements the intuition described briefly in §5.1. In order to conclude that input variables in 𝑉 0 together smoothly affect an output variable 𝑣 in the computation of 𝑐; 𝑐 ′ , the 𝑝 ′′ considers the intermediate state after the first command 𝑐, and forms two groups of variables at that intermediate state: 𝑑 ′ (𝑣) and 𝑝 ′ (𝑣) 𝑐 . Note that the desired smoothness property for the input variables in 𝑉 0 and the output variable 𝑣 may fail if the first command 𝑐 uses some variable 𝑢 0 ∈ 𝑉 0 non-smoothly to update a variable 𝑢 ′ 0 in 𝑑 ′ (𝑣), or it uses some variable 𝑢 1 ∈ 𝑉 0 to compute the value of a variable 𝑢 ′ 1 ∈ 𝑝 ′ (𝑣) 𝑐 . In the former case, the non-smoothness of 𝑐 causes an issue, and in the latter case, the non-smoothness of 𝑐 ′ causes an issue. The 𝑝 ′′ collects the input variables that avoid these two failure modes and also do not influence the termination of the sequence. As we show in our soundness theorem, doing so is sufficient because it amounts to using a version of chain rule for the target smoothness property.

𝑑 (𝑣) ≜                fv(𝑒) ∪ 𝜇=(𝛼,_) ∈Name fv(𝑒 ′ [𝜇/𝑦]) if 𝑣 ≡ 𝑥 fv(𝑒) ∪ {val 𝜇 } ∪ fv(𝑒 ′ [𝜇/𝑦]) if 𝑣 ≡ val 𝜇 for 𝜇 = (𝛼, _) fv(𝑒) ∪ {pr 𝜇 , 𝜇} ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 ) if 𝑣 ≡ pr 𝜇 for 𝜇 = (𝛼, _) fv(𝑒) ∪ {cnt 𝜇 } if 𝑣 ≡ cnt 𝜇 for 𝜇 = (𝛼, _) {𝑣 } otherwise.
The abstract semantics of an if command conservatively assumes that any variable in its condition 𝑏 may affect the value of any output variable (by influencing whether the true or false branch of the command gets executed) and this influence is potentially non-smooth. For every output variable 𝑣, the smooth set 𝑝 ′′ (𝑣) for the if command implements this assumption by excluding free variables in 𝑏, and the computed dependency set does the same but this time by including free variables in 𝑏.

The abstract semantics of a loop computes the least fixed point of a monotone operator 𝐹 ♯ : D ♯ → D ♯ using the standard Kleene iteration. The operator 𝐹 ♯ describes the effect of one iteration of the loop, and it is derived from the standard loop unrolling and our abstract semantics of sequencing and the if command.

The abstract semantics of an observe command obs(dist N (𝑒 1 , 𝑒 2 ), 𝑟 ) uses the fact that the command has the same concrete semantics as the assignment like := like × pdf N (𝑟 ; 𝑒 1 , 𝑒 2 ), where pdf N is the density function of the normal distribution. The semantics computes (𝑝, 𝑑, 𝑉 ) according to that of the assignment, which we explained earlier.

The final case is the abstract semantics of a sample command. The semantics performs a case analysis on the first argument of sam. If it is a constant expression not involving any variables, then the abstract semantics constructs the name 𝜇 of the sampled random variable, and updates 𝑝, 𝑑, and 𝑉 according to the concrete semantics of the command. Otherwise, the abstract semantics acknowledges that the precise name 𝜇 of the random variable cannot be known statically, and performs so called weak update by joining two pieces of information before and after the update of the command in the concrete semantics. Note that the abstract semantics does not require the third argument of sam should be the identity function. The ability of dealing with a general function in the third argument is needed since our analysis is intended to be applied to programs after the transformation of the SPGE, which may introduce such an argument.

The abstract semantics is well-defined under the following relatively weak assumption:

Assumption 1 (Expression analysis and free variables). 𝑒 ♯ ⊇ fv(𝑒) 𝑐 for all expressions 𝑒.

This assumption is satisfied by the instantiations of the semantics with differentiability and local Lipschitzness, which are used in our implementation. It will be assumed in the rest of the paper.

Theorem 5.6. If Assumption 1 holds, then for all commands 𝑐, we have 𝑐 ♯ ∈ D ♯ , that is, when we let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯ , we have 𝑝 (𝑣) ⊇ 𝑑 (𝑣) 𝑐 and 𝑑 (𝑣) ⊇ 𝑉 for all variables 𝑣 ∈ Var.

Example 5.7 (Differentiability). Consider the differentiability property and the example program of Example 5.5. Let (𝑝 1 , 𝑑 1 , 𝑉 1 ) and (𝑝 2 , 𝑑 2 , 𝑉 2 ) be the results of analysing the first assignment command 𝑦 := 𝑥 * 𝑥 and the following if command of the program. Then,

(𝑝 1 , 𝑑 1 , 𝑉 1 ) = (𝜆𝑣.Var, 𝜆𝑣 . (𝑣≡𝑦) ? {𝑥 } : {𝑣 }, ∅), (𝑝 2 , 𝑑 2 , 𝑉 2 ) = (𝜆𝑣.{𝑥 } 𝑐 , 𝜆𝑣 . (𝑣≡𝑠) ? {𝑥 } : {𝑥, 𝑣 }, {𝑥 }).
Let (𝑝, 𝑑, 𝑉 ) be the analysis result for the entire program. Then,

𝑝 (𝑣) = 𝑉 1 ∪ (𝑝 1 ) ∩ (𝑑 2 (𝑣)) 𝑐 ∪ (𝑑 1 ) ∪ (𝑝 2 (𝑣) 𝑐 ) 𝑐 = 𝑑 1 (𝑥) 𝑐 = {𝑥 } 𝑐 , 𝑉 = 𝑉 1 ∪ (𝑑 1 ) ∪ (𝑉 2 ) = {𝑥 }. Also, 𝑑 (𝑣) = 𝑉 1 ∪ (𝑑 1 ) ∪ (𝑑 2 (𝑣)) = {𝑥 }.
As shown in Fig. 3, the variable 𝑥 that may affect the condition expression of the if command is removed from the smoothness sets, and 𝑝 (𝑠) = 𝑝 (𝑦) = {𝑥 } 𝑐 . Note that this result is conservative with respect to 𝑦. □ 5.2.2 Analysis Soundness and Assumptions. The soundness of our analysis states that for every command 𝑐, its abstract semantics 𝑐 ♯ ∈ D ♯ over-approximates the concrete semantics 𝑐 via 𝛾: 𝑐 ∈ 𝛾 ( 𝑐 ♯ ). The soundness is conditioned on Assumption 1 and six new assumptions. One of the new assumptions is about the soundness of 𝑒 ♯ . The remaining five assumptions are concerned with the predicate family for the target smoothness property 𝜙 = (𝜙 𝐾,𝐿 : 𝐾, 𝐿 ⊆ Var), and say that certain canonical operators are smooth according to 𝜙 so that using them in the abstract semantics should not cause an issue. In this subsection, we present the six assumptions one by one, and sketch how those assumptions are related to the soundness.

We start with the assumption that the analysis of each expression 𝑒 ♯ under-approximates the set of variables in which the evaluation of 𝑒 is smooth (and thus over-approximates the smoothness property of 𝑒).

Assumption 2 (Expression analysis soundness). For all expressions 𝑒, variables 𝑥, subsets 𝐾, 𝐿 ⊆ Var, and states

𝜏 ∈ St[Var \ 𝐾] such that 𝐾 = 𝑒 ♯ and 𝐿 = {𝑥 }, if we let 𝑔 : St[𝐾] → St[𝐿] be the function defined by 𝑔(𝜎) ≜ [𝑥 ↦ → 𝑒 (𝜎 ⊕ 𝜏)], the function 𝑔 satisfies 𝜙 𝐾,𝐿 (i.e., 𝑔 ∈ 𝜙 𝐾,𝐿 ).
This assumption is used in our soundness argument whenever the abstract semantics uses 𝑒 ♯ for computing smoothness information about an expression 𝑒.

The next two assumptions assert the smoothness of the standard operators on the product spaces.

Assumption 3 (Projection). For all 𝐾, 𝐿 ⊆ Var with 𝐾 ⊇ 𝐿, the projection 𝜋 𝐾,𝐿 satisfies 𝜙 𝐾,𝐿 .

Assumption 4 (Pairing). For all 𝐾, 𝐿, 𝑀 ⊆ Var with 𝐿 ∩ 𝑀 = ∅, if 𝑓 ∈ 𝜙 𝐾,𝐿 and 𝑔 ∈ 𝜙 𝐾,𝑀 , we have ⟨𝑓 , 𝑔⟩ ∈ 𝜙 𝐾,𝐿∪𝑀 , where ⟨𝑓 , 𝑔⟩ is the pairing of two partial functions:

⟨𝑓 , 𝑔⟩(𝜎) ≜ if (𝜎 ∈ dom(𝑓 ) ∩ dom(𝑔)) then 𝑓 (𝜎) ⊕ 𝑔(𝜎) else undefined. Note that St[𝐿 ∪ 𝑀] is isomorphic to St[𝐿] × St[𝑀]
, the product space that we referred to above. The assumptions say that the projection is smooth, and the pairing of smooth functions is smooth. Our analysis uses Assumption 3 to deal with variables not modified by a command. For instance, when analysing an assignment 𝑥 := 𝑒, the analysis uses Assumption 3 and concludes that on every output variable 𝑣 other than 𝑥, the assignment is smooth in all the input variables. Assumption 4 is used to justify the handling of a sequence 𝑐; 𝑐 ′ by our analysis, in particular, the part that the analysis combines smoothness information over multiple output variables after the first command 𝑐.

The projection and pairing assumptions are about how shrinking and expanding output variables affect the target smoothness property. The next restriction assumption is about shrinking the input variables. It validates the weakening of the 𝐾 part of |= Φ(𝑓 , 𝐾, 𝐿), and is used in the abstract semantics of 𝑐; 𝑐 ′ (and other composite commands).

Assumption 5 (Restriction). For all 𝐾, 𝐾 ′ , 𝐿 ⊆ Var with 𝐾 ⊇ 𝐾 ′ , and

𝜏 ∈ St[𝐾 \ 𝐾 ′ ], if 𝑓 ∈ 𝜙 𝐾,𝐿 , then we have 𝑔 ∈ 𝜙 𝐾 ′ ,𝐿 , where 𝑔(𝜎) ≜ if (𝜎 ⊕ 𝜏 ∈ dom(𝑓 )) then 𝑓 (𝜎 ⊕ 𝜏) else undefined.
The following assumption says that the function composition preserves smoothness. It is related to the chain rule for differentiation, and used to justify the abstract semantics of a sequence 𝑐; 𝑐 ′ . Assumption 6 (Composition). For all 𝐾, 𝐿, 𝑀 ⊆ Var, if 𝑓 ∈ 𝜙 𝐾,𝐿 and𝑔 ∈ 𝜙 𝐿,𝑀 , we have𝑔•𝑓 ∈ 𝜙 𝐾,𝑀 , where 𝑔 • 𝑓 is the standard composition of two partial functions:

(𝑔 • 𝑓 ) (𝜎) ≜ if (𝜎 ∈ dom(𝑓 ) ∧ 𝑓 (𝜎) ∈ dom(𝑔)) then 𝑔(𝑓 (𝜎)) else undefined.
The final assumption lets the analysis infer smoothness information about the completelyundefined function. It is used to justify the handling of loops by our analysis. Assumption 7 (Strictness). For all 𝐾, 𝐿 ⊆ Var, we have

(𝜆𝜎 ∈ St[𝐾].undefined) ∈ 𝜙 𝐾,𝐿 .
Theorem 5.8 (Soundness). If Assumptions 1-7 hold, the analysis computes the sound abstraction of the concrete semantics of commands in the following sense: for all commands 𝑐, 𝑐 ∈ 𝛾 ( 𝑐 ♯ ).

Remark 5.9. A standard method for proving a property of a loop or more generally a recursively defined function is so called Scott induction. In this method, we view a property as a set T of state transformers and a loop as the least fixed point of a continuous function 𝐹 on state transformers. Then, we prove the three conditions: (i) T contains the least state transformer, (ii) it is closed under the least upper bound of any increasing sequence of state transformers, and (iii) T is preserved by 𝐹 . The first and second conditions are called strictness and admissibility, respectively, and these three conditions imply that the least fixed point of 𝐹 belongs to T .

Our soundness proof for the loop case deviates slightly from this standard method. If it followed the method instead, we would need, in addition to the strictness assumption, the following assumption, which corresponds to the second admissibility condition: Assumption 8 (Admissibility). Let 𝐾, 𝐿 ⊆ Var, and order partial functions in

[St[𝐾] ⇀ St[𝐿]]
by the inclusion of the graphs of partial functions. Then, for every increasing sequence

{𝑓 𝑛 : St[𝐾] ⇀ St[𝐿]} 𝑛 ∈N (i.e.
, the graph of 𝑓 𝑛+1 includes that of 𝑓 𝑛 for all 𝑛 ∈ N), if every 𝑓 𝑛 satisfies 𝜙 𝐾,𝐿 , so does the least upper bound 𝑓 ∞ of the sequence (defined by its graph being the union of the graphs of all 𝑓 𝑛 's).

The inclusion of this admissibility assumption would, then, limit the applicability of our program analysis, since some well-known smoothness properties, such as (global) Lipschitz continuity and local boundedness, do not satisfy the assumption, although they satisfy our five assumptions (Assumptions 3-7) (see Table 2). On the plus side, the inclusion of the admissibility assumption could enable our analysis to handle loops more accurately, possibly by tracking the impact of the boolean condition of each loop on smoothness more precisely. Our soundness proof avoids the admissibility assumption by exploiting the fact that our analysis handles loop conditions conservatively: our analysis drops all the variables that loop conditions may depend on from the set of smooth variables, so that it avoids finding too precise inductive predicates that can break soundness. 14□

Instantiations

Our program analysis requires that the family of smoothness predicates should satisfy Assumptions 3-7. Although these assumptions are violated by some smoothness properties, such as partial differentiability and partial continuity, they are met by our leading example 𝜙 (𝑑) for differentiability (Example 5.1), and also by the predicate family 𝜙 (𝑙) for local Lipschitzness, which is used in our implementation. Recall the definitions of the predicate families 𝜙 (𝑑) and 𝜙 (𝑙) : for all 𝐾, 𝐿 ⊆ Var,

𝜙 (𝑑) 𝐾,𝐿 ≜ {𝑓 : St[𝐾] ⇀ St[𝐿] | dom(𝑓 ) is open and 𝑓 is (jointly) differentiable in its domain}, 𝜙 (𝑙) 𝐾,𝐿 ≜ {𝑓 : St[𝐾] ⇀ St[𝐿] | dom(𝑓 )
) -𝑓 (𝜎 1 ) ∥ 2 ≤ 𝐶 ∥𝜎 0 -𝜎 1 ∥ 2 for all 𝜎 0 , 𝜎 1 ∈ 𝑂 }.
Theorem 5.10. Both 𝜙 (𝑑) and 𝜙 (𝑙) satisfy Assumptions 3-7.

Remark 5.11. The requirement of open domain in 𝜙 (𝑑) is sometimes too constraining and hurts the accuracy of the analysis. It can, however, be relaxed, and we can generalise 𝜙 (𝑑) to the following Table 1. Failure cases of the composition assumption. For each given 𝜙, we have 𝑓 , 𝑔 ∈ 𝜙 but 𝑔 • 𝑓 ∉ 𝜙, where 𝑓 and 𝑔 are interpreted as (total or partial) functions from St[𝐾] to St [𝐿]. Let 𝑐 1 ≡ (𝑦 = 𝑥 2 ; 𝑧 = 𝑔(𝑦)) and 𝑐 2 ≡ (𝑦 = 𝑥; 𝑧 = 𝑔(𝑥, 𝑦)). Then, for each 𝑖-th 𝜙, 𝑐 𝑖 ♯ incorrectly concludes that 𝑧 is smooth with respect to 𝑥.

𝜙 𝑓 𝑔

𝜙

(𝑑 ′′ ) 𝐾,𝐿 𝑓 (𝑥) = 𝑥 2 defined on R 𝑔(𝑥) = 1 [𝑥 >0] defined on [0, 1] 𝜙 (pd) 𝐾,𝐿 , 𝜙 (pc) 𝐾,𝐿 𝑓 (𝑥) = (𝑥, 𝑥) defined on R 𝑔(𝑥, 𝑦) = 𝑥𝑦/(𝑥 2 + 𝑦 2 ) if (𝑥, 𝑦) ≠ (0, 0) 0 otherwise defined on R 2
predicate family 𝜙 (𝑑 ′ ) , which corresponds to the standard definition of differentiability on a manifold with boundary in differential geometry [Lee 2012, Chapter 2]:

𝜙 (𝑑 ′ ) 𝐾,𝐿 ≜ {𝑓 : St[𝐾] ⇀ St[𝐿] | for all 𝜎 ∈ dom(𝑓 ), there exist an open 𝑈 ⊆ St[𝐾] and 𝑔 : 𝑈 → St[𝐿] such that 𝜎 ∈ 𝑈 , 𝑓 = 𝑔 on 𝑈 ∩ dom(𝑓 ), and 𝑔 is (jointly) differentiable}.
Note the weakening of open-domain requirement in 𝜙 (𝑑 ′ ) : the open domain 𝑈 in the above definition does not have to be included in dom(𝑓 ). The family 𝜙 (𝑑 ′ ) satisfies Assumptions 3-7, and can lead to a more permissive instantiation of our program analysis than the family 𝜙 (𝑑) , especially in handling atomic commands, such as assignment, sample, and observe. We point out that 𝜙 (𝑑 ′ ) does not satisfy Assumption 8 (i.e., the admissibility assumption), while 𝜙 (𝑑) does satisfy it. As a result, if the handling of loops in our analysis is changed such that loop conditions are analysed more accurately, the analysis may remain sound only for 𝜙 (𝑑) , not for 𝜙 (𝑑 ′ ) , as explained in Remark 5.9. □ Remark 5.12. At this point, the reader might feel that Assumptions 3-7 are satisfied by nearly all smoothness properties. This impression is not accurate. For instance, the composition assumption does not hold for the notions of differentiability of partial functions formalised by the following 𝜙 (𝑑 ′′ ) and 𝜙 (pd) , nor for the partial continuity formalised by 𝜙 (pc) : Table 1 contains counterexamples that show the failure of the composition assumption for these predicate families. In fact, when instantiated with these families, our program analysis is not sound. The same table shows example programs and incorrect conclusions derived by our analysis.

𝜙 (𝑑 ′′ ) 𝐾,𝐿 ≜ {𝑓 : St[𝐾] ⇀ St[𝐿] | 𝑓 is (jointly) differentiable
Table 2 shows more target smoothness properties from mathematics, and whether each property satisfies Assumptions 3-7 (and Assumption 8). Recall that our program analysis does not require Assumption 8 for soundness; the table shows the assumption just for reference. The target properties from "cont." to "real analytic" in the table (and three more) satisfy Assumptions 3-7, so that our analysis can be immediately applied to those target properties while remaining sound. □

ALGORITHM FOR THE SPGE VARIABLE-SELECTION PROBLEM

We now put together the results from §4 and §5 to formally define and soundly (yet approximately) solve the SPGE variable-selection problem. We start with the formal definition of the problem: Definition 6.1 (SPGE Variable-Selection Problem; Formal). Assume we are given a model 𝑐 𝑚 , a guide 𝑐 𝑔 , and a (initial) simple reparameterisation plan 𝜋 0 such that 𝑐 𝑚 , 𝑐 𝑔 , and 𝑐 𝑔 𝜋 always terminate and have no double-sampling errors for all 𝜋 ⊑ 𝜋 0 . Here we write 𝜋 ⊑ 𝜋 ′ if the graph of 𝜋 is included in that of 𝜋 ′ . Given these 𝑐 𝑚 , 𝑐 𝑔 , and 𝜋 0 , find a reparameterisation plan 𝜋 ⊑ 𝜋 0 such that (i) 𝜋 is simple and satisfies (R2) and (R3) in §4.2, and (ii) |rv(𝜋)| is maximised. We say that 𝜋 is a sound solution if it satisfies (i), and an optimal solution if it satisfies (i) and (ii). □

Table 2. Various well-known target smoothness properties from mathematics, and whether each of them satisfies Assumptions 3-7 (and Assumption 8). Here "cont. " and "diff. " denote "continuous" and "differentiable". The properties above the double horizontal line are defined such that they include the open-domain requirement as in 𝜙 (𝑑) and 𝜙 (𝑙) , and the properties below the line are defined without the open-domain requirement.

Target smoothness property A3 (proj.) A4 (pair.) A5 (rest.) A6 (comp.) A7 (stri.) A8 (admi.) cont. (C 0 )

• • • • • • locally Lipschitz (= 𝜙 (𝑙) ) • • • • • • uniformly cont. • • • • • × Lipschitz cont. • • • • • × diff. (= 𝜙 (𝑑) ) • • • • • • continuously diff. (C 1 ) • • • • • • smooth (C ∞ ) • • • • • • real analytic (C 𝜔 ) • • • • • • partially cont. (= 𝜙 (𝑝𝑐) ) • • • × • • partially diff. (= 𝜙 (𝑝𝑑) ) • • • × • • almost-everywhere cont. • • × × • • almost-everywhere diff. • • × × • • coordinatewise non-decreasing • • • • • • locally bounded • • • • • × bounded × • • • • × Borel measurable • • • • • • locally integrable • • × × • × integrable × • × × • ×
The input 𝜋 0 in the problem is a newcomer. It fixes a semantics-preserving transformation for all the sample commands. Typically, 𝜋 0 is defined on the entire NameEx × DistEx × LamEx, and remains fixed across all input model-guide pairs (𝑐 𝑚 , 𝑐 𝑔 ). More importantly, it is valid so that the change of any sample command by 𝜋 0 preserves the semantics of the command when we take into account both the second distribution argument and the third lambda argument of the sample command. The validity of 𝜋 0 is inherited by any sound solution 𝜋 of the SPGE variable-selection problem since validity as a property on reparameterisation plans is down-closed with respect to the ⊑ order. In our setup, 𝜋 0 is fixed to be the following reparameterisation plan from §4.1:

𝜋 0 (𝑛, dist N (𝑒 1 , 𝑒 2 ), 𝜆𝑦.𝑒 3 ) ≜ (dist N (0, 1), 𝜆𝑦.𝑒 3 [(𝑦 × √ 𝑒 2 + 𝑒 1 )/𝑦]) (11)
for all 𝑛 ∈ NameEx and expressions 𝑒 1 , 𝑒 2 , and 𝑒 3 .

As an example of the SPGE variable-selection problem, consider the problem for the 𝜋 0 in Eq. ( 11) and the model-guide pair (𝑐 𝑚 , 𝑐 𝑔 ) given in Fig. 1, where "z i " in the figure is interpreted as name("z i ", 0). Then, as discussed in §2, the problem has the following optimal solution:

𝜋 ≜ 𝜋 0 | 𝑆 × DistEx × LamEx for 𝑆 ≜ {name(𝛼, 𝑒) ∈ NameEx | 𝛼 "z 2 "}.
We present an algorithm for computing a sound (yet possibly suboptimal) solution to the problem.

(1) By running our program analysis instantiated with differentiability (described in §5.2 and §5.3), compute

(p 𝑚 , d 𝑚 , V 𝑚 ) ≜ 𝑐 𝑚 ♯ and (p 𝑔 , d 𝑔 , V 𝑔 ) ≜ 𝑐 𝑔 ♯
, where we use p, d, and V for the output of the analysis to distinguish them from densities 𝑝 and distributions 𝑑.

(2) Using p 𝑚 and p 𝑔 , check

𝜃 ⊆ 𝐾, where 𝐾 ≜ p 𝑚 (like) ∩ 𝜇 ∈Name p 𝑚 (pr 𝜇 ) ∩ 𝜇 ∈Name p 𝑔 (pr 𝜇 ). ( 12
)
If the check fails, return an error message that our analysis cannot discharge (R2) for any 𝜋, since the analysis concludes that the density function of 𝑐 𝑚 or 𝑐 𝑔 can be non-differentiable in 𝜃 (even when rv(𝜋) = ∅). If the check passes, initialise the set of reparameterised random variables by

𝑆 ≜ {(𝛼, 𝑖) ∈ Name | for all 𝑖 ′ ∈ N, (𝛼, 𝑖 ′ ) ∈ Name =⇒ (𝛼, 𝑖 ′ ) ∈ 𝐾 }.
(

If the check passes, return 𝜋 as the output of the algorithm. If not, update𝑆 by𝑆 \{(𝛼, 𝑖) ∈ Name} after choosing some (𝛼, _) ∈ 𝑆, and then repeat the above procedure (from the step (3), the point where we construct 𝜋 using 𝑆) until 𝑆 becomes empty. Our algorithm computes a sound solution (in the sense stated in Definition 6.1), partly because of the soundness of our program analysis: Theorem 6.2. Let 𝑐 𝑚 , 𝑐 𝑔 , and 𝜋 0 be the inputs to the SPGE variable-selection problem. If the above algorithm returns 𝜋 for (𝑐 𝑚 , 𝑐 𝑔 , 𝜋 0 ), then 𝜋 is a sound solution for the problem.

We point out that the theorem holds for the local Lipschitzness case as well (in addition to the differentiability case). That is, if the above algorithm runs our program analysis instantiated with local Lipschitzness (instead of differentiability), and if it returns an output 𝜋, then 𝜋 is a sound solution to the SPGE variable-selection problem that uses (R2') and (R3') (instead of (R2) and (R3)).

Our algorithm solves the problem only approximately: there is no formal guarantee that it always computes an optimal solution. The suboptimality may arise due to two approximations: the overapproximation of our program analysis when it computes differentiability (or local Lipschitzness) information, and the heuristic choices made by our algorithm when the algorithm computes the random-variable set 𝑆. We demonstrate, however, that our algorithm finds optimal solutions for all the benchmarks in §7.

Our algorithm calls our program analysis at most |{𝛼 ∈ Str | (𝛼, _) ∈ 𝑆 0 }| + 2 times, where 𝑆 0 is the initial value of 𝑆 (i.e., the set of random variables whose sample commands are to be transformed) in the algorithm. However, for all the benchmarks in §7, our algorithm terminated with the initial set 𝑆 0 and thus called our analysis only 3 times (on the model, the guide, and the reparameterised guide according to 𝑆 0 ). Based on these results and our intuition on the algorithm, we conjecture that our algorithm always terminates with the initial set 𝑆 0 under a mild condition on 𝜋 0 and our analysis (e.g., 𝑒 ♯ is computed inductively on 𝑒). Since the conjecture is still open, our algorithm might not succeed in the first iteration, and if so, it continues to search for a sound solution greedily. Note that there are many other ways to continue the search and our algorithm uses just one of them (as it is linear-time).

EXPERIMENTAL EVALUATION

In our experiments, we consider two research questions. First, can the analysis proposed in §5 be instantiated and implemented so that it can produce meaningful smoothness results on real-world probabilistic programs? Second, can the algorithm proposed in §6 find near-optimal solutions to the SPGE variable-selection problem on real-world probabilistic programs? To assess the two questions, we have implemented a static smoothness analyser for Pyro programs based on §5, and a variable selector based on §6 which (approximately) solves the variable-selection problem. 15 Our analyser and variable selector are implemented in OCaml, and support a subset of the Pyro PPL and two smoothness properties: differentiability and local Lipschitzness.

Implementation. Although the analysis described in §5 may look simple when considering a basic PPL, real-world PPLs such as Pyro are of a much higher degree of complexity. First, they provide a large panel of continuous/discrete probability distributions for sample and observe commands, and library functions for tensors and neural networks. Second, programs in real-world PPLs may fail to be smooth for reasons other than if-else and while commands. In particular, values sampled from discrete distributions, and arguments to operators and distribution constructors that are well-defined only on a strict subset of values, may induce non-smoothness. A straightforward treatment of these will result in an overly conservative analysis, treating far too many variables as potentially non-smooth. Third, Pyro programs typically rely on tensors (of large, statically unknown size) to deal with large datasets, and it is generally infeasible to reason about each (real-valued) element of tensors individually. In the following, we discuss how our static analyser addresses these issues and provides sound, useful information about smoothness of Pyro programs. Distributions and library functions. Our analyser supports 17 distributions (continuous or discrete). Each distribution is characterized by a pair (𝑏, 𝑎) for a boolean 𝑏 and an array of booleans 𝑎, where 𝑏 (or 𝑎 𝑖 ) denotes whether its probability density is differentiable or locally Lipschitz with respect to the sampled value (or the 𝑖-th argument) of the distribution. For example, a normal distribution is described by (true,[true,true]) (assuming that the second argument is positive) and a Poisson distribution by (false,[true]). Similarly, the analyser supports a large number of PyTorch/Pyro library functions for tensors and neural networks, and assumes the correct smoothness information about them. For instance, the ReLU function is considered locally Lipschitz but not differentiable. Refining smoothness information based on safety pre-analysis. Although the expression x/y is generally non-smooth with respect to y (even if it is well-defined for y=0), if more information is available, for instance that y always lies in range [1, 10], we can safely consider it smooth with respect to both x and y. Likewise, the density of a normal distribution is generally non-smooth with respect to the standard deviation argument 𝜎 (even if it is well-defined for 𝜎 ≤ 0), so more precise smoothness information can be produced when 𝜎 is known to be always positive. Thus, establishing precise smoothness information requires to first establish safety properties related to program operations. To achieve this, our tool actually performs two analyses in sequence: (i) a safety pre-analysis infers ranges over all numerical variables and marks each argument to an operator or a distribution constructor as either "safe" or "potentially unsafe"; (ii) the program analysis formalised in §5 utilises information computed in the first phase to produce precise smoothness information. The first phase boils down to a forward abstract interpretation based on basic abstract domains like intervals and signs [START_REF] Cousot | Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints[END_REF]. It logs safety information for each program statement just like static analyses for runtime errors and undefined behaviors [START_REF] Blanchet | A Static Analyzer for Large Safety Critical Software[END_REF]]. As formalised in §5.2, the second analysis is compositional. Due to their different nature, the two analyses need to be done in sequence. Tensors. Pyro programs commonly use nested loops and indexed tensors. As the number of scalar values in each tensor is often statically unknown (or known but huge), treating each scalar as a separate variable is not feasible; so we rely on a conservative summarisation of tensors. Intuitively, we treat all scalars in a tensor as a "block": e.g., when density might not be smooth with respect to some scalar(s) of a tensor, the analysis conservatively concludes that it might not be smooth with respect to all scalars in the tensor. In our experiments, this abstraction does not result in any precision loss.

Evaluation. We evaluated our analyser and variable selector on 13 representative Pyro examples from the Pyro webpage [Uber AI Labs 2022] that use standard SVI engines and contain explicitly written model-guide pairs (without AutoGuide). They include advanced models with deep neural networks such as attend-infer-repeat [START_REF] Ali Eslami | Attend, Infer, Repeat: Fast Scene Understanding with Generative Models[END_REF]] and single-cell annotation using variational inference [START_REF] Xu | Probabilistic harmonization and annotation of single-cell transcriptomics data with deep generative models[END_REF]. Additionally, we included the example in Fig. 1, for which Pyro offers an unsound reparameterisation plan. Smoothness analyser. We assess our smoothness analyser on the 14 Pyro examples for differentiability and local Lipschitzness ( §5.3), and show a subset of results in Table 4 (see §I for the rest). The results demonstrate that our analysis can cope successfully with real-world Pyro programs. First, our analysis is accurate. For all examples, the analysis identifies the exact ground-truth set of random variables and parameters in which the density of the program is differentiable (or locally Lipschitz). In many of them, information computed by the pre-analysis is required to achieve these exact results; e.g., some examples (e.g., dpmm and air) require precise information about which distribution arguments can be proved to be always in the proper range of values. Second, the runtime of our analysis is low. Typical probabilistic programming applications are not of a very large size, and conceptual complexity is generally the main issue, thus the analysis performance presents no scalability concern. We draw two more observations from the results. First, for spnor-m and air-g, the density of each program is not locally Lipschitz in one continuous random variable. These non-local-Lipschitznesses arise as follows: for the former, the random variable ("z 2 " in Fig. 1) is used in the branch condition of an if-else command that contains observe commands, thereby creating discontinuity; and for the latter, the random variable ("z_where") is passed into the denominator of a division operator, thereby causing a division-by-zero error for some value.

Second, for all the other examples, the density is locally Lipschitz in all continuous random variables and parameters, but is often non-differentiable in many parameters (and continuous random Table 5. Results of variable selections. "Ours-Time" denote the runtime of our variable selector in seconds. "Ours-Sound" and "Pyro \ Ours" denote the number of random variables in the example that are in 𝜋 ours , and that are in 𝜋 0 but not in 𝜋 ours , respectively, where 𝜋 ours and 𝜋 0 denote the reparameterisation plans given by our variable selector and by Pyro. "Pyro \ Ours" is partitioned into "Sound" and "Unsound": the latter denotes the number of random variables that make (R2') or (R3') violated when added to 𝜋 ours , and the former denotes the number of the rest. "#CR" and "#DR" denote the total number of continuous and discrete random variables in the example. We consider {(𝛼, 𝑖) ∈ Name} as one random variable for each 𝛼 ∈ Name. See §I for the rest of results.

Ours

Pyro The results demonstrate that for all examples, our variable selector finds the optimal reparameterisation plan with a small runtime. We also observe that for all cases, it terminates in the first iteration and calls our smoothness analyser only three times, as mentioned in §6. Note that the reparameterisation plan given by Pyro is also optimal for all but two examples. We emphasise, however, that our variable selector not only finds a reparameterisation plan but also verifies the local Lipschitzness requirements (R2') and (R3'), whereas Pyro's default variable selector does not do so. Indeed, for two examples, Pyro's reparameterisation plan is unsound as it violates the local Lipschitzness requirements. Hence, these results should be interpreted as: for all but two examples, our variable selector (and smoothness analyser) successfully verifies that the default gradient estimator used by Pyro satisfies important smoothness-related preconditions for unbiasedness, namely the local Lipschitzness requirements.

The two examples for which Pyro becomes unsound are spnor and air. Recall that they have two continuous random variables (one for each) in which their densities are not locally Lipschitz. The unsoundness of Pyro on these examples stems precisely from the fact that it reparameterises the two non-locally-Lipschitz random variables without checking any local Lipschitzness requirements.

RELATED WORK

The high-level idea of using program transformation for improved posterior inference and model learning in PPLs has been explored previously [START_REF] Claret | Bayesian inference using data flow analysis[END_REF][START_REF] Gorinova | Automatic Reparameterisation of Probabilistic Programs[END_REF][START_REF] Nori | R2: An Efficient MCMC Sampler for Probabilistic Programs[END_REF]Ritchie et al. 2016b;[START_REF] Schulman | Gradient Estimation Using Stochastic Computation Graphs[END_REF]]. In particular, [START_REF] Schulman | Gradient Estimation Using Stochastic Computation Graphs[END_REF] proposed a method for implementing the SPGE for stochastic computation graphs via graph transformation, and this method was adopted in the implementation of the same estimator in Pyro and also in our work. However, the method lacks a formal analysis on the implemented estimator especially in the context of probabilistic programs; it does not have a version of Theorem 4.5, which formally identifies requirements for the unbiasedness of the estimator. Also, the method does not check the required smoothness properties of given probabilistic programs. Our work fills in these gaps. [START_REF] Gorinova | Automatic Reparameterisation of Probabilistic Programs[END_REF] proposed an automatic technique to transform models in a PPL using the same or closely-related transformation of sample commands in the SPGE. The work is, however, concerned with transforming models and taming their posterior distributions, while ours focuses on transforming guides. Also, the work does not check smoothness properties of transformed models that are required for running efficient inference algorithms, such as Hamiltonian Monte Carlo, on those models, while our work checks those properties using our program analysis.

Program analyses or type systems for PPLs have been developed to detect common errors [START_REF] Lee | Towards verified stochastic variational inference for probabilistic programs[END_REF][START_REF] Lew | Trace types and denotational semantics for sound programmable inference in probabilistic languages[END_REF][START_REF] Wang | Sound probabilistic inference via guide types[END_REF], infer important probabilistic properties such as conditional independence [START_REF] Gorinova | Conditional Independence by Typing[END_REF], or automatically plan inference algorithms [WebPPL 2019] as in our work. In particular, WebPPL runs a simple program analysis (checking if there are interleaving sample and observe commands) to decide if it is worth applying sequential Monte Carlo.

The smoothness properties computed by our program analysis, such as differentiability and local Lipschitzness, fall in the scope of hyperliveness in the hierarchy of hyperproperties [START_REF] Clarkson | Hyperproperties[END_REF]. Intuitively, hyperliveness properties are those that cannot be refuted based on any finite counterexample (i.e., made of finitely-many finite execution traces), and counterexamples for differentiability and local Lipschitzness should indeed require infinitely-many execution traces due to the use of limit or all neighbouring inputs in their definitions. Not so many analyses have considered such hyperliveness properties. Among those, the most relevant to our work are the continuity analyses of [START_REF] Chaudhuri | Continuity analysis of programs[END_REF][START_REF] Chaudhuri | Continuity and robustness of programs[END_REF]. It uses a program abstraction that is rather similar to ours, but their analyses suffer from soundness issues, partly due to the incorrect joining of continuity sets [START_REF] Chaudhuri | Continuity analysis of programs[END_REF]] and also to an unsound rule for sequential composition [START_REF] Chaudhuri | Continuity and robustness of programs[END_REF]] (see §A for details). We do not claim that these issues are difficult to fix. Our point is just that developing program analyses for smoothness properties requires special care. Chaudhuri et al.'s work focuses on proving smoothness properties of control software, or revealing the unexpected continuity of discrete algorithms. On the other hand, our program analysis is designed to assist variational inference and model learning for probabilistic programs. [START_REF] Barthe | On the Versatility of Open Logical Relations -Continuity, Automatic Differentiation, and a Containment Theorem[END_REF] proposed a refinement type system, which considers a higher-order functional language and ensures that every typable first-order program is continuous in all variables. On the other hand, our program analysis considers a first-order imperative language and can prove that a program is continuous in some (not necessarily all) variables. Other existing program analyses for smoothness properties include [START_REF] Laurel | A dual number abstraction for static analysis of Clarke Jacobians[END_REF]] which over-approximates the Clarke generalised Jacobian, and [START_REF] Mangal | Probabilistic Lipschitz Analysis of Neural Networks[END_REF]] which proves probabilistic Lipschitzness.

A DEFERRED RESULTS IN §1

A.1 Unsoundness of Continuity Analyses in [START_REF] Chaudhuri | Continuity analysis of programs[END_REF][START_REF] Chaudhuri | Continuity and robustness of programs[END_REF] The continuity analysis in [START_REF] Chaudhuri | Continuity analysis of programs[END_REF]] considers joint continuity, whereas the continuity analysis in [START_REF] Chaudhuri | Continuity and robustness of programs[END_REF]] considers partial continuity. That is, given a command 𝑐, an output variable 𝑣 of 𝑐, and some input variables 𝑢 1 , . . . , 𝑢 𝑚 to 𝑐, the former analyses whether 𝑣 is continuous in {𝑢 1 , . . . , 𝑢 𝑚 } jointly, whereas the latter analyses whether 𝑣 is continuous in 𝑢 𝑖 separately for every 1 ≤ 𝑖 ≤ 𝑚.

Join and Sequence rules. The former analysis contains a rule called Join [Chaudhuri et al. 2010, Figure 3], and the latter analysis contains a rule called Sequence [START_REF] Chaudhuri | Continuity and robustness of programs[END_REF], Figure 1]. The two rules can be rewritten (with some simplifications) as follows, in terms of functions between R 𝑛 : for any 𝑓 , 𝑔 : R 𝑛 → R 𝑛 and 𝑆, 𝑆 ′ ,𝑇 , 𝑈 ⊆ {1, . . . , 𝑛},

For each 𝑗 ∈ 𝑇 , 𝑓 𝑗 is continuous in {𝑥 𝑖 | 𝑖 ∈ 𝑆 } For each 𝑗 ∈ 𝑇 , 𝑓 𝑗 is continuous in {𝑥 𝑖 | 𝑖 ∈ 𝑆 ′ } For each 𝑗 ∈ 𝑇 , 𝑓 𝑗 is continuous in {𝑥 𝑖 | 𝑖 ∈ 𝑆 ∪ 𝑆 ′ } (Join) For each 𝑗 ∈ 𝑇 , 𝑓 𝑗 is continuous in 𝑥 𝑖 for each 𝑖 ∈ 𝑆 For each 𝑘 ∈ 𝑈 , 𝑔 𝑘 is continuous in 𝑦 𝑗 for each 𝑗 ∈ 𝑇 For each 𝑘 ∈ 𝑈 , (𝑔 • 𝑓 ) 𝑘 is continuous in 𝑥 𝑖 for each 𝑖 ∈ 𝑆 (Sequence)
where 𝑓 and 𝑔 are functions of variables 𝑥 1 , . . . , 𝑥 𝑛 and 𝑦 1 , . . . , 𝑦 𝑛 , respectively, and ℎ 𝑖 ≜ proj 𝑖 • ℎ for ℎ : R 𝑛 → R 𝑛 and 𝑖 ∈ {1, . . . , 𝑛} denotes the 𝑖-th component of ℎ. As mentioned above, the Join rule analyses joint continuity, while the Sequence rule analyses partial continuity. Further, the Join rule says that joint continuity is preserved under the union of input variables, while the Sequence rule says that partial continuity is preserved under the composition of functions.

The two rules, however, are unsound with the following counterexamples. Let ℎ : R 2 → R 2 be the function

ℎ(𝑥 1 , 𝑥 2 ) ≜ (𝑥 1 𝑥 2 /(𝑥 2 1 + 𝑥 2 2 ), 𝑥 2 ) if (𝑥 1 , 𝑥 2 ) ≠ (0, 0) (0, 𝑥 2 )
otherwise.

Note that ℎ 1 is continuous in 𝑥 1 and in 𝑥 2 separately, but not in {𝑥 1 , 𝑥 2 } jointly. First, for the Join rule, consider the following 𝑓 : R 2 → R 2 and 𝑆, 𝑆 ′ ,𝑇 ⊆ {1, 2}:

𝑓 (𝑥 1 , 𝑥 2 ) ≜ ℎ(𝑥 1 , 𝑥 2 ), 𝑆 ≜ {1}, 𝑆 ′ ≜ {2}, 𝑇 ≜ {1, 2}.
Then, the premise of the Join rule holds, and so the conclusion of the rule must hold. But this is not the case since 𝑓 1 = ℎ 1 is not continuous in {𝑥 1 , 𝑥 2 }. Hence, the Join rule is unsound. Next, for the Sequence rule, consider the following 𝑓 , 𝑔 : R 2 → R 2 and 𝑆,𝑇 , 𝑈 ⊆ {1, 2}:

𝑓 (𝑥 1 , 𝑥 2 ) ≜ (𝑥 1 , 𝑥 1 ), 𝑔(𝑥 1 , 𝑥 2 ) ≜ ℎ(𝑥 1 , 𝑥 2 ), 𝑆 ≜ 𝑇 ≜ 𝑈 ≜ {1, 2}.
Then, the premise of the Sequence rule holds, and so the conclusion of the rule must hold. But this is not the case since

(𝑔 • 𝑓 ) 1 is not continuous in 𝑥 1 (due to (𝑔 • 𝑓 ) 1 (𝑥 1 , 𝑥 2 ) = 1 [𝑥 1 ≠0] • 1 2 )
. Hence, the Sequence rule is unsound. These counterexamples show that joint continuity is not preserved under the union of input variables, and partial continuity is not preserved under the composition of functions.

The two aforementioned counterexamples can be easily translated into programs: the first becomes 𝑐 1 ≡ (𝑧 := ℎ 1 (𝑥, 𝑦)) and the second becomes 𝑐 2 ≡ (𝑦 := 𝑥; 𝑧 := ℎ 1 (𝑥, 𝑦)), where 𝑥, 𝑦, and 𝑧 are program variables and ℎ 1 is the binary operator defined above. The analysis in [START_REF] Chaudhuri | Continuity analysis of programs[END_REF] deduces that in 𝑐 1 , 𝑧 is continuous in 𝑥 and 𝑦 (jointly), and the analysis in [START_REF] Chaudhuri | Continuity and robustness of programs[END_REF] deduces that in 𝑐 2 , 𝑧 is continuous in 𝑥 (separately). Both deductions, however, are incorrect as seen above, and the two analyses are thus unsound.

Sequence rule (again). Both of the two analyses in [START_REF] Chaudhuri | Continuity analysis of programs[END_REF][START_REF] Chaudhuri | Continuity and robustness of programs[END_REF] contain the Sequence rule (discussed above) which has the following rule as an instance: for all variables 𝑣 0 , 𝑣 1 , 𝑣 2 that are not necessarily distinct, and for all commands 𝑐 1 , 𝑐 2 ,

In 𝑐 1 , 𝑣 1 is continuous in 𝑣 0 In 𝑐 2 , 𝑣 2 is continuous in 𝑣 1 In (𝑐 1 ; 𝑐 2 ), 𝑣 2 is continuous in 𝑣 0
The above instance of the Sequence rule is, however, unsound because it incorrectly handles the dependencies between variables. For instance, consider commands 𝑐 1 ≡ (if (𝑥 < 0) {𝑦 := 0} else {𝑦 := 1}) and 𝑐 2 ≡ (𝑧 := 𝑥 + 𝑦), and variables 𝑣 0 ≡ 𝑥, 𝑣 1 ≡ 𝑥, and 𝑣 2 ≡ 𝑧. Then, in (𝑐 1 ; 𝑐 2 ), 𝑧 is not continuous in 𝑥 (because 𝑧 after (𝑐 1 ; 𝑐 2 ) is 𝑥 if 𝑥 < 0, and is 𝑥 + 1 if 𝑥 ≥ 0). However, the premise of the above rule holds for this case, and so the rule incorrectly concludes that 𝑧 is continuous in 𝑥 under (𝑐 1 ; 𝑐 2 ), by ignoring the dependency of 𝑧 on 𝑦 (which is discontinuous in 𝑥).

Loop rule. The analysis in [START_REF] Chaudhuri | Continuity analysis of programs[END_REF]] contains a rule called Simple-loop (Figure 5 in the paper) to analyse loops, and the analysis in [START_REF] Chaudhuri | Continuity and robustness of programs[END_REF]] contains a rule called Loop (Figure 1 in the paper) which is essentially the same as the Simple-loop rule.

The two rules, however, incorrectly assume that (joint or partial) continuity without any restriction on the domain of functions satisfies the admissibility assumption discussed in Remark 5.9; and this in turn makes the rules unsound. To illustrate the unsoundness, consider a command 𝑐 ≡ (while (0 < 𝑥 < 1) {𝑐 ′ }) with 𝑐 ′ ≡ (𝑦 := 𝑦 + 𝑓 (𝑥); 𝑥 := 𝑔(𝑥)), where 𝑓 and 𝑔 are the continuous operators defined by:

𝑓 (𝑥) ≜          𝑥 if 0 < 𝑥 ≤ 1/2 1 -𝑥 if 1/2 < 𝑥 ≤ 1 0 otherwise, 𝑔(𝑥) ≜          0 if 𝑥 ≤ 0 2𝑥 if 0 < 𝑥 ≤ 1/2 1 otherwise.
Then, the premises of the two rules are satisfied, mainly because 𝑥 and 𝑦 after 𝑐 ′ are continuous jointly in 𝑥 and 𝑦 before 𝑐 ′ , and 𝑥 and 𝑦 do not change in 𝑐 ′ if 𝑥 = 0 or 𝑥 = 1 (i.e., at the "boundary" of the loop condition of 𝑐). Hence, the two rules conclude that 𝑥 and 𝑦 after 𝑐 are continuous in 𝑥 and 𝑦 before 𝑐 (jointly or separately). However, this is an unsound conclusion because in 𝑐, 𝑦 is not continuous in 𝑥 at 𝑥 = 0: we have

𝑦 ′ = 𝑦 if 𝑥 = 0, but 𝑦 ′ → 𝑦 + 1 as 𝑥 → 0 + (more precisely, 𝑦 ′ = 𝑦 if 𝑥 ≤ 0 or 𝑥 ≥ 1, and 𝑦 ′ = 𝑦 + (1 -𝑥) if 0 < 𝑥 < 1)
, where 𝑥 ′ and 𝑦 ′ denote the values of 𝑥 and 𝑦 after 𝑐.

B DEFERRED RESULTS IN §2

B.1 Table Summarising §2

(𝑐) : St[PVar] × St[Name] × St[AVar] → [0, ∞), prs(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝜎 𝑎 ) ≜          𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝜎 𝑎 ) (like) • 𝜇 ∈Name 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝜎 𝑎 ) (pr 𝜇 ) if noerr (𝑐, 𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝜎 𝑎 ) 0 otherwise, vals(𝑐) : St[PVar] × St[Name] × St[AVar] → St[Name], vals(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝜎 𝑎 ) ≜ 𝜆𝜇 ∈ Name. 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝜎 𝑎 ) (val 𝜇 ) if noerr (𝑐, 𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝜎 𝑎 ) 𝜆𝜇 ∈ Name. 0 otherwise,
where noerr (𝑐, 𝜎) is a predicate for a command 𝑐 and 𝜎 ∈ St, defined by noerr (𝑐, 𝜎)

⇐⇒ 𝑐 𝜎 ∈ St ∧ ∀𝜇 ∈ Name. 𝑐 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) ≤ 1 .
The predicate noerr (𝑐, 𝜎) says that 𝑐 terminates for 𝜎 without a double-sampling error. The functions prs and vals generalise the density function 𝑝 and the value function 𝑣, respectively; in particular, they do not assume a particular initial state 𝜎 0 used in Eq. ( 3). We consider the generalisation of 𝑝 and 𝑣 so as to enable inductive proofs.

Although generalising 𝑝 and 𝑣, the functions prs and vals are not sufficient to enable inductive proofs since their inputs and outputs contain some unnecessary parts, which stops induction from working well (especially in the sequential composition case): namely, the part of St[Name] that is not read during execution, and the part of St[AVar] that is not updated during execution. To exclude those unnecessary parts, we first define the set of substates of St[Name] as follows:

𝜉 𝑛 ∈ St □ [Name] ≜ 𝐾 ⊆Name St[𝐾].
Based on these substates, we define the next functions:

prs □ (𝑐) : St[PVar] × St □ [Name] → [0, ∞), prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ≜          𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (like) • 𝜇 ∈dom(𝜉 𝑛 ) 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (pr 𝜇 ) if ∃𝜎 𝑟 . used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) 0 otherwise, vals □ (𝑐) : St[PVar] × St □ [Name] → St □ [Name], vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ≜ 𝜆𝜇 ∈ dom(𝜉 𝑛 ). 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (val 𝜇 ) if ∃𝜎 𝑟 . used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) 𝜆𝜇 ∈ dom(𝜉 𝑛 ). 0 otherwise, pvars □ (𝑐) : St[PVar] × St □ [Name] → St[PVar], pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ≜ 𝜆𝑥 ∈ PVar. 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (𝑥) if ∃𝜎 𝑟 .
(like) = 1 ∧ 𝜉 𝑛 = 𝜎 | dom(𝜉 𝑛 ) ∧ dom(𝜉 𝑛 ) = {𝜇 ∈ Name | 𝑐 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 1} .
The predicate used (𝑐, 𝜎, 𝜉 𝑛 ) says that 𝑐 terminates for 𝜎 without a double-sampling error, like is initialised to 1 in 𝜎, and 𝜉 𝑛 is the Name part of 𝜎 that is sampled during the execution of 𝑐 from 𝜎. By using used (-, -, -), the three functions do not take the unnecessary part of a state as an input, and do not return the unnecessary part of a state in the output. The three functions are well-defined.

Lemma C.1. prs □ , vals □ , and pvars □ are well-defined, i.e., they do not depend on the choice of 𝜎 𝑟 .

We now state two main lemmas for Theorem 4.2. The first lemma describes how prs and vals are connected with prs □ and vals □ . The second lemma says that a particular integral involving prs □ , vals □ , and pvars □ is the same for 𝑐 and 𝑐 𝜋 if a reparameterisation plan 𝜋 is valid.

Lemma C.2. Let 𝑐 be a command, and 𝑓 𝑖 : R → R for 𝑖 ∈ {1, 2, 3} be measurable functions such that 𝑓 1 (𝑟 ) ≥ 0 for all 𝑟 ∈ R. Define 𝑓 * :

St[Name] → St[AVar] by 𝑓 * (𝜎 𝑛 ) (𝑎) ≜              1 if 𝑎 ≡ like 𝑓 1 (𝜎 𝑛 (𝜇)) if 𝑎 ≡ pr 𝜇 for 𝜇 ∈ Name 𝑓 2 (𝜎 𝑛 (𝜇)) if 𝑎 ≡ val 𝜇 for 𝜇 ∈ Name 𝑓 3 (𝜎 𝑛 (𝜇)) if 𝑎 ≡ cnt 𝜇 for 𝜇 ∈ Name.
Then, for all 𝜎 𝑝 ∈ St[PVar] and all measurable ℎ :

St[Name] → R, ∫ 𝑑𝜎 𝑛 prs(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) • ℎ vals(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 )
where the integral on the LHS is defined if and only if the one on the RHS is defined, and the function

𝑔 : St □ [Name] → R is defined by 𝑔(𝜉 ′′ 𝑛 ) = ∫ 𝑑𝜉 ′ 𝑛 1 [dom(𝜉 ′′ 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • 𝜇 ∈dom(𝜉 ′ 𝑛 ) 𝑓 1 (𝜉 ′ 𝑛 (𝜇)) • ℎ 𝜉 ′′ 𝑛 ⊕ 𝜆𝜇 ∈ dom(𝜉 ′ 𝑛 ). 𝑓 2 (𝜉 ′ 𝑛 (𝜇)) . (14)
Lemma C.3. Let 𝑐 be a command and 𝑔 :

St[PVar] × St □ [Name] → R be a measurable function. Then, for all 𝜎 𝑝 ∈ St[PVar], ∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 )
where the integral on the LHS is defined if and only if the one on the RHS is defined.

We now prove Theorem 4.2 using these two lemmas.

Proof of Theorem 4.2. Let 𝜋 be a valid reparameterisation plan, 𝑐 be a command, 𝜎 𝜃 ∈ St[𝜃 ], and ℎ : St[Name] → R be a measurable function. Suppose that the integral on the LHS of Theorem 4.2 is defined. Recall that for a given 𝜎 𝑛 ∈ St[Name], the definitions of 𝑝 and 𝑣 (in §3 and §4.1) use the initial state 𝜎 ≜ 𝜎 𝜃 ⊕ 𝜎 𝑛 ⊕ 𝜎 0 ∈ St, where 𝜎 0 ∈ St[(PVar \ 𝜃 ) ∪ AVar] depends on 𝜎 𝑛 and has the following definition:

𝜎 0 (𝑣) ≜                    0 if 𝑣 ∈ PVar \ 𝜃 1 if 𝑣 ≡ like N (𝜎 𝑛 (𝜇); 0, 1) if 𝑣 ≡ pr 𝜇 for 𝜇 ∈ Name 𝜎 𝑛 (𝜇) if 𝑣 ≡ val 𝜇 for 𝜇 ∈ Name 0 if 𝑣 ≡ cnt 𝜇 for 𝜇 ∈ Name.
The initial state 𝜎 can be re-expressed as

𝜎 = 𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝑓 * (𝜎 𝑛 ) (15
𝜎 𝑝 (𝑥) ≜ 𝜎 𝜃 (𝑥) if 𝑥 ∈ 𝜃 0 if 𝑥 ∈ PVar \ 𝜃, 𝑓 * (𝜎 𝑛 ) (𝑎) ≜              1 if 𝑎 ≡ like 𝑓 1 (𝜎 𝑛 (𝜇)) if 𝑎 ≡ pr 𝜇 for 𝜇 ∈ Name 𝑓 2 (𝜎 𝑛 (𝜇)) if 𝑎 ≡ val 𝜇 for 𝜇 ∈ Name 𝑓 3 (𝜎 𝑛 (𝜇)) if 𝑎 ≡ cnt 𝜇 for 𝜇 ∈ Name,
where 𝑓 1 (𝑟 ) ≜ N (𝑟 ; 0, 1), 𝑓 2 (𝑟 ) ≜ 𝑟 , and 𝑓 3 (𝑟 ) ≜ 0. Using this, we get the desired equation:

∫ 𝑑𝜎 𝑛 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ) • ℎ 𝑣 𝑐,𝜎 𝜃 (𝜎 𝑛 ) = ∫ 𝑑𝜎 𝑛 prs(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) • ℎ vals(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜎 𝑛 prs(𝑐 𝜋 ) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) • ℎ vals(𝑐 𝜋 ) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • ℎ 𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )
where 𝑔 : St □ [Name] → R is defined as Eq. ( 14). The first equality holds by Eq. ( 15) and the definition of 𝑝 We first prove that the following equations hold for any measurable ℎ ′ :

St[Name] → R: ∫ 𝑑𝜎 𝑛 1 [noerr (𝑐,𝜎 𝑝 ⊕𝜎 𝑛 ⊕𝑓 * (𝜎 𝑛 )) ] • ℎ ′ (𝜎 𝑛 ) = ∫ 𝑑𝜎 𝑛 1 [noerr (𝑐,𝜎 𝑝 ⊕𝜎 𝑛 ⊕𝑓 * (𝜎 𝑛 )) ] • ℎ ′ (𝜎 𝑛 ) • 𝐾 ⊆Name 1 [used (𝑐,𝜎 𝑝 ⊕𝜎 𝑛 ⊕𝑓 * (𝜎 𝑛 ),𝜎 𝑛 | 𝐾 ) ] = 𝐾 ⊆Name ∫ 𝑑𝜎 𝑛 1 [used (𝑐,𝜎 𝑝 ⊕𝜎 𝑛 ⊕𝑓 * (𝜎 𝑛 ),𝜎 𝑛 | 𝐾 ) ] • 1 [noerr (𝑐,𝜎 𝑝 ⊕𝜎 𝑛 ⊕𝑓 * (𝜎 𝑛 )) ] • ℎ ′ (𝜎 𝑛 ) = 𝐾 ⊆Name ∫ [𝐾→R] 𝑑𝜉 𝑛 ∫ [Name\𝐾→R] 𝑑𝜉 ′ 𝑛 1 [used (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ),𝜉 𝑛 ) ] • 1 [noerr (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 )) ] • ℎ ′ (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 ) = 𝐾 ⊆Name ∫ [𝐾→R] 𝑑𝜉 𝑛 𝐿 ⊆Name ∫ [𝐿→R] 𝑑𝜉 ′ 𝑛 1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • 1 [used (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ),𝜉 𝑛 ) ] • 1 [noerr (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 )) ] • ℎ ′ (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 ) = ∫ 𝑑𝜉 𝑛 ∫ 𝑑𝜉 ′ 𝑛 1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • 1 [used (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ),𝜉 𝑛 ) ] • 1 [noerr (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 )) ] • ℎ ′ (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 ) .
All of these equations mean that one side of the equation is defined if and only if the other side is defined, and when both sides are defined, they are the same.

)) = ∫ 𝑑𝜉 𝑛 ∫ 𝑑𝜉 ′ 𝑛 1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • 1 [used (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ • 1 [noerr (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 )) ] • prs(𝑐) (𝜎 𝑝 , 𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 , 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 )) • ℎ vals(𝑐) (𝜎 𝑝 , 𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 , 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 )) = ∫ 𝑑𝜉 𝑛 ∫ 𝑑𝜉 ′ 𝑛 1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • 1 [used (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ),𝜉 𝑛 ) ] • prs(𝑐) (𝜎 𝑝 , 𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 , 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 )) • ℎ vals(𝑐) (𝜎 𝑝 , 𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 , 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 )) = ∫ 𝑑𝜉 𝑛 ∫ 𝑑𝜉 ′ 𝑛 1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • 1 [used (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ),𝜉 𝑛 ) ] • prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝜇 ∈dom(𝜉 ′ 𝑛 ) 𝑓 1 (𝜉 ′ 𝑛 (𝜇)) • ℎ vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ⊕ 𝜆𝜇 ∈ dom(𝜉 ′ 𝑛 ). 𝑓 2 (𝜉 ′ 𝑛 (𝜇)) = ∫ 𝑑𝜉 𝑛 ∫ 𝑑𝜉 ′ 𝑛 1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝜇 ∈dom(𝜉 ′ 𝑛 ) 𝑓 1 (𝜉 ′ 𝑛 (𝜇)) • ℎ vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ⊕ 𝜆𝜇 ∈ dom(𝜉 ′ 𝑛 ). 𝑓 2 (𝜉 ′ 𝑛 (𝜇)) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • ∫ 𝑑𝜉 ′ 𝑛 1 [dom(vals □ (𝑐) (𝜎 𝑝 ,𝜉 𝑛 ))⊎dom(𝜉 ′ 𝑛 )=Name] • 𝜇 ∈dom(𝜉 ′ 𝑛 ) 𝑓 1 (𝜉 ′ 𝑛 (𝜇)) • ℎ vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ⊕ 𝜆𝜇 ∈ dom(𝜉 ′ 𝑛 ). 𝑓 2 (𝜉 ′ 𝑛 (𝜇)) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 )
where 𝑔 : St □ [Name] → R is defined as in the statement of this lemma, and each equation again means that one side of it is defined if and only if the other side is defined, and when both sides are defined, they are the same. The first and third equalities hold because prs(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) ≠ 0 implies 1 [noerr (𝑐,𝜎 𝑝 ⊕𝜎 𝑛 ⊕𝑓 * (𝜎 𝑛 )) ] = 1. The second equality uses the equation that we have shown in the previous paragraph. The fourth equality holds because of the following reason: if

1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • 1 [used (𝑐,𝜎 𝑝 ⊕𝜉 𝑛 ⊕𝜎 𝑟 ,𝜉 𝑛 ) ] = 1 for 𝜎 𝑟 ≜ 𝜉 ′ 𝑛 ⊕ 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 ), then prs(𝑐) (𝜎 𝑝 , 𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 , 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 )) = 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (like) • 𝜇 ∈Name 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (pr 𝜇 ) = 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (like) • 𝜇 ∈dom(𝜉 𝑛 ) 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (pr 𝜇 ) • 𝜇 ∈Name\dom(𝜉 𝑛 ) (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (pr 𝜇 ) = prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝜇 ∈Name\dom(𝜉 𝑛 ) (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (pr 𝜇 ) = prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝜇 ∈dom(𝜉 ′ 𝑛 ) 𝑓 1 (𝜉 ′ 𝑛 (𝜇)) and vals(𝑐) (𝜎 𝑝 , 𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 , 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 )) = 𝜆𝜇 ∈ Name. 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (val 𝜇 ) = 𝜆𝜇 ∈ dom(𝜉 𝑛 ). 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (val 𝜇 ) ⊕ 𝜆𝜇 ∈ Name \ dom(𝜉 𝑛 ). (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (val 𝜇 ) = vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ⊕ 𝜆𝜇 ∈ Name \ dom(𝜉 ′ 𝑛 ). (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (val 𝜇 ) = vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ⊕ 𝜆𝜇 ∈ dom(𝜉 ′ 𝑛 ). 𝑓 2 (𝜉 ′ 𝑛 (𝜇)
) . These equalities for prs(𝑐) and vals(𝑐) themselves hold for the below reasons:

• The first equalities hold by used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) and the definitions of prs(𝑐) and vals(𝑐).

• The second equalities hold by Lemma C.6-(2), which is applicable since used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ).

• The third equalities hold by used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) and the definitions of prs □ (𝑐) and vals □ (𝑐).

• The fourth equalities hold by dom(𝜉 𝑛 ) ⊎ dom(𝜉 ′ 𝑛 ) = Name and the definition of 𝑓 * . Returning back to the main equations, we point out that the fifth equality comes from the next fact:

1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] • prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ≠ 0 =⇒ 1 [used (𝑐,𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ) ⊕𝑓 * (𝜉 𝑛 ⊕𝜉 ′ 𝑛 ),𝜉 𝑛 ) ] = 1.
The justification for this implication is given below: for all 𝜇 ∈ Name.

Proof. We prove the lemma by induction on the structure of 𝑐. Let 𝜎 ∈ St such that 𝑐 𝜎 ∈ St.

Cases 𝑐 ≡ skip, or 𝑐 ≡ (𝑥 := 𝑒), or 𝑐 ≡ obs(𝑑, 𝑟 ). In these cases, 𝑐 𝜎 (cnt 𝜇 ) = 𝜎 (cnt 𝜇 ) and 𝑐 𝜎 (𝜇) = 𝜇 for all 𝜇. The claim of the lemma, thus, follows.

Case 𝑐 ≡ (𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒 ′ )). Let 𝜇 ≜ 𝑛 𝜎, 𝑝 ≜ 𝑑 𝜎, and 𝑟 ≜ 𝑒 ′ [𝜇/𝑦] 𝜎. Then,

𝑐 𝜎 = 𝜎 [𝑥 ↦ → 𝑟, val 𝜇 ↦ → 𝑟, pr 𝜇 ↦ → 𝑝 (𝑟 ), cnt 𝜇 ↦ → 𝜎 (cnt 𝜇 ) + 1].
Thus, the claim of the lemma follows.

Case 𝑐 ≡ (𝑐 ′ ; 𝑐 ′′ ). Pick 𝜇 ∈ Name. Then,

𝑐 ′ ; 𝑐 ′′ 𝜎 (𝜇) = 𝑐 ′′ ( 𝑐 ′ 𝜎) (𝜇) = 𝑐 ′ 𝜎 (𝜇) = 𝜎 (𝜇).
Here the second and third equalities use induction hypothesis on 𝑐 ′ and 𝑐 ′′ , respectively. Also,

𝑐 ′ ; 𝑐 ′′ 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 𝑐 ′′ ( 𝑐 ′ 𝜎) (cnt 𝜇 ) -𝑐 ′ 𝜎 (cnt 𝜇 ) + 𝑐 ′ 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) ≥ 0.
The inequality here uses induction hypothesis on 𝑐 ′ and 𝑐 ′′ .

Case 𝑐 ≡ (if 𝑏 {𝑐 ′ } else {𝑐 ′′ }). Assume that 𝑏 𝜎 = true. We will prove the claims of the lemma under this assumption. The other case of 𝑏 = false can be proved similarly. Pick 𝜇 ∈ Name. Then, by induction hypothesis on 𝑐 ′ , 𝑐 𝜎 (𝜇) = 𝑐 ′ 𝜎 (𝜇) = 𝜎 (𝜇) and 𝑐 𝜎 (cnt 𝜇 ) = ′ 𝜎 (cnt 𝜇 ) ≥ 𝜎 (cnt 𝜇 ).

Case 𝑐 ≡ (while 𝑏 {𝑐 ′ }). Let T be the following subset of [St → St ⊥ ]:

𝑓 ∈ T ⇐⇒ ∀𝜎 ∈ St. 𝑓 (𝜎) ≠ ⊥ =⇒ ∀𝜇 ∈ Name. 𝑓 (𝜎) (𝜇) = 𝜎 (𝜇) ∧ 𝑓 (𝜎) (cnt 𝜇 ) ≥ 𝜎 (cnt 𝜇 ) .
Let 𝐹 be the operator on [St → St ⊥ ] whose least fixed point becomes the semantics of the loop 𝑐. The desired conclusion follows if we show that T contains 𝜆𝜎.⊥ and is closed under taking the limit of a chain in T , and 𝐹 preserves T . The least element 𝜆𝜎.⊥ belongs to T since there are no states 𝜎 with (𝜆𝜎.⊥) (𝜎) ≠ ⊥. Consider an increasing sequence 𝑓 0 , 𝑓 1 , . . . in T , and let 𝑓 ∞ ≜ 𝑛 ∈N 𝑓 𝑛 . Pick 𝜎 such that 𝑓 ∞ (𝜎) ≠ ⊥. Then, 𝑓 ∞ (𝜎) = 𝑓 𝑚 (𝜎) for some 𝑚 ∈ N. Since 𝑓 𝑚 ∈ T , we have

𝑓 𝑚 (𝜎) (𝜇) = 𝜎 (𝜇) and 𝑓 𝑚 (𝜎) (cnt 𝜇 ) ≥ 𝜎 (cnt 𝜇 )
for all 𝜇 ∈ Name. Since 𝑓 𝑚 (𝜎) = 𝑓 ∞ (𝜎), we also have, for every 𝜇 ∈ Name, 𝑓 ∞ (𝜎) (𝜇) = 𝜎 (𝜇) and 𝑓 ∞ (𝜎) (cnt 𝜇 ) ≥ 𝜎 (cnt 𝜇 ), as desired. It remains to show that 𝐹 (𝑓 ) ∈ T for all 𝑓 ∈ T . Pick 𝑓 ∈ T and 𝜎 ∈ St such that 𝐹 (𝑓 ) (𝜎) ∈ St. If 𝑏 𝜎 = false, we have 𝐹 (𝑓 ) (𝜎) = 𝜎, and the claims of the lemma follow. Otherwise, 𝐹 (𝑓 ) (𝜎) = 𝑓 ( 𝑐 ′ 𝜎). Pick 𝜇 ∈ Name. Then, by induction hypothesis on 𝑐 ′ and the membership 𝑓 ∈ T ,

𝐹 (𝑓 ) (𝜎) (𝜇) = 𝑓 ( 𝑐 ′ 𝜎) (𝜇) = 𝑐 ′ 𝜎 (𝜇) = 𝜎 (𝜇),
and

𝐹 (𝑓 ) (𝜎) (cnt 𝜇 ) = 𝑓 ( 𝑐 ′ 𝜎) (cnt 𝜇 ) ≥ 𝑐 ′ 𝜎 (cnt 𝜇 ) ≥ 𝜎 (cnt 𝜇 ).
We have just shown that 𝐹 (𝑓 ) ∈ T , as desired. □ Definition C.5. Define used -as the predicate used but without the condition that like should be 1. That is, for all commands 𝑐, states 𝜎 ∈ St, and

𝜉 𝑛 ∈ St □ [Name], used -(𝑐, 𝜎, 𝜉 𝑛 ) ⇐⇒ 𝑐 𝜎 ∈ St ∧ 𝑐 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) ≤ 1 for all 𝜇 ∈ Name ∧ 𝜉 𝑛 = 𝜎 | dom(𝜉 𝑛 ) ∧ dom(𝜉 𝑛 ) = {𝜇 ∈ Name | 𝑐 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 1}.
Lemma C.6. Let 𝑐 be a command, 𝜎 0 , 𝜎 1 ∈ St, and 𝜉 𝑛 ∈ St □ [Name]. Suppose that used -(𝑐, 𝜎 0 , 𝜉 𝑛 ) and 𝜎 1 | 𝑉 = 𝜎 0 | 𝑉 for 𝑉 ≜ PVar ∪ dom(𝜉 𝑛 ). Then, the following properties hold:

(1) 𝑐 𝜎 1 ∈ St. (2) 𝑐 𝜎 1 (𝑎) = 𝜎 1 (𝑎) for all 𝑎 ∈ {pr 𝜇 , val 𝜇 , cnt 𝜇 | 𝜇 ∈ Name \ dom(𝜉 𝑛 )}. (3) 𝑐 𝜎 1 (𝑣) = 𝑐 𝜎 0 (𝑣) for all 𝑣 ∈ PVar ∪ {pr 𝜇 , val 𝜇 | 𝜇 ∈ dom(𝜉 𝑛 )}. (4) 𝑐 𝜎 1 (like) = 𝑐 𝜎 0 (like), if 𝜎 0 (like) = 𝜎 1 (like). (5) 𝑐 𝜎 1 (𝑎) -𝜎 1 (𝑎) = 𝑐 𝜎 0 (𝑎) -𝜎 0 (𝑎) for all 𝑎 ∈ {cnt 𝜇 | 𝜇 ∈ Name}. Proof. For 𝜎 ′ 0 , 𝜎 ′ 1 ∈ St and 𝜉 ′ 𝑛 ∈ St □ [Name], write 𝜎 ′ 0 ∼ 𝜉 ′ 𝑛 𝜎 ′ 1 to mean that 𝜎 ′ 0 | 𝑉 = 𝜎 ′ 1 | 𝑉 for 𝑉 ≜ PVar ∪ dom(𝜉 ′ 𝑛 ).
Note that using this notation, we can write the conditions of the lemma as follows:

used -(𝑐, 𝜎 0 , 𝜉 𝑛 ) ∧ 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 .
We will prove, by induction on the structure of 𝑐, that these conditions imply the five properties claimed by the lemma. Our proof will sometimes use a simple observation that the five properties claimed by the lemma and the relationship 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 imply used -(𝑐, 𝜎 1 , 𝜉 𝑛 ). One consequence of the observation is that if our lemma holds, its five properties also hold with 𝜎 0 and 𝜎 1 swapped. We will often use this consequence.

Case 𝑐 ≡ skip. In this case, 𝑐 𝜎 0 = 𝜎 0 and 𝑐 𝜎 1 = 𝜎 1 . From these equalities, the claimed properties (1), ( 2), ( 4) and ( 5) follow. For the remaining property (3), we note that dom(𝜉 𝑛 ) = ∅ and the property, thus, follows from 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 .

Case 𝑐 ≡ (𝑥 := 𝑒). In this case, 𝑐 𝜎 0 = 𝜎 0 [𝑥 ↦ → 𝑒 𝜎 0 ] and 𝑐 𝜎 1 = 𝜎 1 [𝑥 ↦ → 𝑒 𝜎 1 ]. The results are not ⊥, and they are identical to the pre-states 𝜎 0 and 𝜎 1 as far as auxiliary variables in AVar are concerned. Also, expressions in commands do not depend on variables other than program variables, so that 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 gives 𝑒 𝜎 0 = 𝑒 𝜎 1 and 𝑐 𝜎 0 (𝑥) = 𝑐 𝜎 1 (𝑥) for all 𝑥 ∈ PVar. From all of these observations, the claimed properties (1)-( 5) follow.

Case 𝑐 ≡ (𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒 ′ )). Since 𝜎 0 (𝑥) = 𝜎 1 (𝑥) for all 𝑥 ∈ PVar, we have 𝑛 𝜎 0 = 𝑛 𝜎 1 and 𝑑 𝜎 0 = 𝑑 𝜎 1 . Let 𝜇 ≜ 𝑛 𝜎 0 , 𝑝 ≜ 𝑑 𝜎 0 , and 𝑟 ≜ 𝑒 ′ [𝜇/𝑦] 𝜎 0 . By the semantics of the sample commands, we have

𝑐 𝜎 0 = 𝜎 0 [𝑥 ↦ → 𝑟, val 𝜇 ↦ → 𝑟, pr 𝜇 ↦ → 𝑝 (𝜎 0 (𝜇)), cnt 𝜇 ↦ → 𝜎 0 (cnt 𝜇 ) + 1]. Since used -(𝑐, 𝜎 0 , 𝜉 𝑛 ) holds, we have 𝜉 𝑛 = 𝜎 0 | {𝜇 } , which in turn implies 𝜎 0 (𝜇) = 𝜎 1 (𝜇) because 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 . Thus, 𝑒 ′ [𝜇/𝑦] 𝜎 1 = 𝑒 ′ [𝜇/𝑦] 𝜎 0 = 𝑟 , and 𝑐 𝜎 1 = 𝜎 1 [𝑥 ↦ → 𝑒 ′ [𝜇/𝑦] 𝜎 1 , val 𝜇 ↦ → 𝑒 ′ [𝜇/𝑦] 𝜎 1 , pr 𝜇 ↦ → 𝑝 (𝜎 1 (𝜇)), cnt 𝜇 ↦ → 𝜎 1 (cnt 𝜇 ) + 1] = 𝜎 1 [𝑥 ↦ → 𝑟, val 𝜇 ↦ → 𝑟, pr 𝜇 ↦ → 𝑝 (𝜎 0 (𝜇)), cnt 𝜇 ↦ → 𝜎 1 (cnt 𝜇 ) + 1].
The RHS of the last equality implies that the five properties claimed by the lemma hold.

Case 𝑐 ≡ obs(𝑑, 𝑟 ). We have 𝑑 𝜎 0 = 𝑑 𝜎 1 since 𝜎 0 (𝑥) = 𝜎 1 (𝑥) for all 𝑥 ∈ PVar. Let 𝑝 ≜ 𝑑 𝜎 0 . Then,

𝑐 𝜎 0 = 𝜎 0 [like ↦ → 𝜎 0 (like) • 𝑝 (𝑟 )] and 𝑐 𝜎 1 = 𝜎 1 [like ↦ → 𝜎 1 (like) • 𝑝 (𝑟 )].
Also, dom(𝜉 𝑛 ) = ∅ since used -(𝑐, 𝜎 0 , 𝜉 𝑛 ) holds. From what we have proved and also the agreement of 𝜎 0 and 𝜎 1 on program variables, the five properties claimed by the lemma follow.

Case 𝑐 ≡ (𝑐 ′ ; 𝑐 ′′ ). Since 𝑐 𝜎 0 = 𝑐 ′′ † ( 𝑐 ′ 𝜎 0 ) ∈ St, we have 𝑐 ′ 𝜎 0 ∈ St. Let 𝜎 ′ 0 ≜ 𝑐 ′ 𝜎 0 , 𝑁 0 ≜ {𝜇 ∈ Name | 𝑐 ′′ 𝜎 ′ 0 (cnt 𝜇 ) -𝜎 0 (cnt 𝜇 ) = 1}, 𝑁 ′ 0 ≜ {𝜇 ∈ Name | 𝜎 ′ 0 (cnt 𝜇 ) -𝜎 0 (cnt 𝜇 ) = 1}.
Then, 𝑁 0 = dom(𝜉 𝑛 ) because used -(𝑐 ′ ; 𝑐 ′′ , 𝜎 0 , 𝜉 𝑛 ) holds. We will prove the following facts:

(1)

𝑁 ′ 0 ⊆ 𝑁 0 . (2) Let 𝜉 ′ 𝑛 ≜ 𝜉 𝑛 | 𝑁 ′ 0 , and 𝜉 ′′ 𝑛 ≜ 𝜉 𝑛 | (𝑁 0 \𝑁 ′ 0 ) . Then, used -(𝑐 ′ , 𝜎 0 , 𝜉 ′ 𝑛 ) and used -(𝑐 ′′ , 𝜎 ′ 0 , 𝜉 ′′ 𝑛 ) hold. (3) 𝑐 ′ 𝜎 1 ∈ St. (4) Let 𝜎 ′ 1 ≜ 𝑐 ′ 𝜎 1 . Then, 𝜎 ′ 0 ∼ 𝜉 ′′ 𝑛 𝜎 ′ 1 .
These four facts imply the five properties claimed by the lemma. Here is the reason. Note that

𝜎 0 ∼ 𝜉 ′ 𝑛 𝜎 1 since dom(𝜉 ′ 𝑛 ) = 𝑁 ′ 0 ⊆ 𝑁 0 = dom(𝜉 𝑛 )
. This relationship between 𝜎 0 and 𝜎 1 and the second fact let us use induction hypothesis on (𝑐 ′ , 𝜉 ′ 𝑛 , 𝜎 0 , 𝜎 1 ). Also, the second and fourth facts allow us to use induction hypothesis on (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ). We can derive the five properties from what we get from these two applications of induction hypothesis:

(1) By induction hypothesis on (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ), we have

𝑐 ′ ; 𝑐 ′′ 𝜎 1 = 𝑐 ′′ 𝜎 ′ 1 ∈ St. (2) For all 𝑎 ∈ {pr 𝜇 , val 𝜇 , cnt 𝜇 | 𝜇 ∈ Name \ dom(𝜉 𝑛 )}, 𝑐 ′ ; 𝑐 ′′ 𝜎 1 (𝑎) = 𝑐 ′′ 𝜎 ′ 1 (𝑎) = 𝜎 ′ 1 (𝑎) = 𝑐 ′ 𝜎 1 (𝑎) = 𝜎 1 (𝑎).
Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

The second equality comes from induction hypothesis on (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ) and dom(𝜉 ′′ 𝑛 ) ⊆ dom(𝜉 𝑛 ), and the fourth equality from induction hypothesis on (𝑐 ′ , 𝜉 ′ 𝑛 , 𝜎 0 , 𝜎 1 ) and dom(𝜉 ′ 𝑛 ) ⊆ dom(𝜉 𝑛 ).

(3) For all 𝑣 ∈ PVar ∪ {pr 𝜇 , val 𝜇 | 𝜇 ∈ dom(𝜉 ′′ 𝑛 )}, by induction hypothesis on (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ),

𝑐 ′ ; 𝑐 ′′ 𝜎 1 (𝑣) = 𝑐 ′′ 𝜎 ′ 1 (𝑣) = 𝑐 ′′ 𝜎 ′ 0 (𝑣) = 𝑐 ′ ; 𝑐 ′′ 𝜎 0 (𝑣). Also,
𝑐 ′ ; 𝑐 ′′ 𝜎 1 (𝑎) = 𝑐 ′′ 𝜎 ′ 1 (𝑎) = 𝜎 ′ 1 (𝑎) = 𝑐 ′ 𝜎 1 (𝑎) = 𝑐 ′ 𝜎 0 (𝑎) = 𝜎 ′ 0 (𝑎) = 𝑐 ′′ 𝜎 ′ 0 (𝑎) = 𝑐 ′ ; 𝑐 ′′ 𝜎 0 (𝑎).
The second equality uses induction hypothesis on (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ), and the fourth comes from the induction hypothesis on (𝑐 ′ , 𝜉 ′ 𝑛 , 𝜎 0 , 𝜎 1 ). The sixth equality follows from induction hypothesis applied to (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ) and again to the same tuple but with 𝜎 ′ 0 and 𝜎 ′ 1 swapped. (4) If 𝜎 0 (like) = 𝜎 1 (like), by induction hypothesis on (𝑐 ′ , 𝜉 ′ 𝑛 , 𝜎 0 , 𝜎 1 ),

𝜎 ′ 0 (like) = 𝑐 ′ 𝜎 0 (like) = 𝑐 ′ 𝜎 1 (like) = 𝜎 ′ 1 (like), which in turn implies, by induction hypothesis on (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ), 𝑐 ′ ; 𝑐 ′′ 𝜎 0 (like) = 𝑐 ′′ 𝜎 ′ 0 (like) = 𝑐 ′′ 𝜎 ′ 1 (like) = 𝑐 ′ ; 𝑐 ′′ 𝜎 1 (like). (5) For all 𝑎 ∈ {cnt 𝜇 | 𝜇 ∈ Name}, 𝑐 ′ ; 𝑐 ′′ 𝜎 1 (𝑎) -𝜎 1 (𝑎) = 𝑐 ′ ; 𝑐 ′′ 𝜎 1 (𝑎) -𝑐 ′ 𝜎 1 (𝑎) + 𝑐 ′ 𝜎 1 (𝑎) -𝜎 1 (𝑎) = 𝑐 ′′ 𝜎 ′ 1 (𝑎) -𝜎 ′ 1 (𝑎) + 𝑐 ′ 𝜎 1 (𝑎) -𝜎 1 (𝑎) = 𝑐 ′′ 𝜎 ′ 0 (𝑎) -𝜎 ′ 0 (𝑎) + 𝑐 ′ 𝜎 0 (𝑎) -𝜎 0 (𝑎) = 𝑐 ′ ; 𝑐 ′′ 𝜎 0 (𝑎) -𝜎 0 (𝑎).
The only non-trivial inequality is the third one, and it follows from induction hypothesis on (𝑐 ′ , 𝜉 ′ 𝑛 , 𝜎 0 , 𝜎 1 ) and (𝑐 ′′ , 𝜉 ′′ 𝑛 , 𝜎 ′ 0 , 𝜎 ′ 1 ). We prove the four facts as follows:

(1) Let 𝜇 ∈ 𝑁 ′ 0 . Since used -(𝑐 ′ ; 𝑐 ′′ , 𝜎 0 , 𝜉 𝑛 ), we have 𝑐 ′′ 𝜎 ′ 0 (𝜇) -𝜎 0 (𝜇) = 𝑐 ′ ; 𝑐 ′′ 𝜎 0 -𝜎 0 (𝜇) ≤ 1. Also, by Lemma C.4 and the definition of 𝑁 ′ 0 , 𝑐 ′′ 𝜎 ′ 0 (𝜇) -𝜎 0 (𝜇) ≥ 𝜎 ′ 0 (𝜇) -𝜎 0 (𝜇) = 1. Thus, 𝑐 ′′ 𝜎 ′ 0 (𝜇) -𝜎 0 (𝜇) = 1, which implies that 𝜇 ∈ 𝑁 0 , as desired.

(2) We should show that used -(𝑐 ′ , 𝜎 0 , 𝜉 ′ 𝑛 ) and used -(𝑐 ′′ , 𝜎 ′ 0 , 𝜉 ′′ 𝑛 ) hold. The conjuncts in the definition of used -(𝑐 ′ , 𝜎 0 , 𝜉 ′ 𝑛 ) except the second follow immediately from used -(𝑐 ′ ; 𝑐 ′′ , 𝜎 0 , 𝜉 𝑛 ) and the definition of 𝜉 𝑛 . For the remaining second conjunct, we use Lemma C.4 and used -(𝑐 ′ ; 𝑐 ′′ , 𝜎 0 , 𝜉 𝑛 ), and prove the conjunct as shown below: for all 𝜇 ∈ Name,

𝑐 ′ 𝜎 0 (cnt 𝜇 ) -𝜎 0 (cnt 𝜇 ) ≤ 𝑐 ′′ ( 𝑐 ′ 𝜎 0 ) (cnt 𝜇 ) -𝜎 0 (cnt 𝜇 ) ≤ 1.
For used -(𝑐 ′′ , 𝜎 ′ 0 , 𝜉 ′′ 𝑛 ), we first note that the first and third conjuncts in its definition are direct consequences of used -(𝑐 ′ ; 𝑐 ′′ , 𝜎 0 , 𝜉 𝑛 ) and the definition of 𝜉 ′′ 𝑛 . We prove the second conjunct in the definition as follows: for all 𝜇 ∈ Name,

𝑐 ′′ 𝜎 ′ 0 (cnt 𝜇 ) -𝜎 ′ 0 (cnt 𝜇 ) = 𝑐 ′ ; 𝑐 ′′ 𝜎 0 (cnt 𝜇 ) -𝑐 ′ cnt 0 (cnt 𝜇 ) ≤ 𝑐 ′ ; 𝑐 ′′ 𝜎 0 (cnt 𝜇 ) -cnt 0 (cnt 𝜇 ) ≤ 1.
The first inequality uses Lemma C.4, and the second comes from used -(𝑐 ′ ; 𝑐 ′′ , 𝜎 0 , 𝜉 𝑛 ). It remains to show the fourth conjunct in the definition of used -(𝑐 ′′ , 𝜎 ′ 0 , 𝜉 ′′ 𝑛 ), which we do below: for all 𝜇 ∈ Name,

𝑐 ′′ 𝜎 ′ 0 (cnt 𝜇 ) -𝜎 ′ 0 (cnt 𝜇 ) = 1 ⇐⇒ 𝑐 ′′ 𝜎 ′ 0 (cnt 𝜇 ) -𝜎 ′ 0 (cnt 𝜇 ) = 1 ∧ 𝜎 ′ 0 (cnt 𝜇 ) -𝜎 0 (cnt 𝜇 ) = 0 ⇐⇒ 𝜇 ∈ 𝑁 0 ∧ 𝜇 ∉ 𝑁 ′ 0 ⇐⇒ 𝜇 ∈ dom(𝜉 ′′ 𝑛 ).
The first equivalence comes from Lemma C.4 and 𝑐 ′′ 𝜎 ′ 0 (cnt 𝜇 ) -𝜎 0 (cnt 𝜇 ) ≤ 1, which holds because of used -(𝑐 ′ ; 𝑐 ′′ , 𝜎 0 , 𝜉 𝑛 ). The second equivalence follows from the definitions of 𝑁 0 and 𝑁 ′ 0 .

(3) Since 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 implies 𝜎 0 ∼ 𝜉 ′ 𝑛 𝜎 1 and we have used -(𝑐 ′ , 𝜎 0 , 𝜉 ′ 𝑛 ), we can apply induction hypothesis to (𝑐 ′ , 𝜉 ′ 𝑛 , 𝜎 0 , 𝜎 1 ), and get 𝑐 ′ 𝜎 1 ∈ St. (4) We continue our reasoning in the previous item, and derive from induction hypothesis on (𝑐 ′ , 𝜉 ′ 𝑛 , 𝜎 0 , 𝜎 1 ) the fact that for all 𝑥 ∈ PVar,

𝜎 ′ 0 (𝑥) = 𝑐 ′ 𝜎 0 (𝑥) = 𝑐 ′ 𝜎 1 (𝑥) = 𝜎 ′ 1 (𝑥). Also, for all 𝜇 ∈ dom(𝜉 ′′ 𝑛 ), 𝜎 ′ 0 (𝜇) = 𝜎 0 (𝜇) = 𝜎 1 (𝜇) = 𝜎 ′ 1 (𝜇)
, where the first and third equalities come from Lemma C.4, and the second equality follows from the assumption that 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 .

Case 𝑐 ≡ (if 𝑏 {𝑐 ′ } else 𝑐 ′′ ). Assume that 𝑏 𝜎 0 = true. Then, 𝑐 𝜎 0 = 𝑐 ′ 𝜎 0 . We prove the five properties claimed by the lemma under this assumption. The proof for the other possibility, namely, 𝑏 𝜎 0 = false is similar. Since 𝜎 0 ∼ 𝜉 𝑛 𝜎 1 , the states 𝜎 0 and 𝜎 1 coincide for the values of program variables. Thus, 𝑏 𝜎 1 = true, and 𝑐 𝜎 1 = 𝑐 ′ 𝜎 1 . Since 𝑐 𝜎 0 = 𝑐 ′ 𝜎 0 as well, it suffices to show the five properties claimed by the lemma for (𝑐 ′ , 𝜎 0 , 𝜎 1 , 𝜉 𝑛 ). This sufficient condition follows from induction hypothesis on (𝑐 ′ , 𝜎 0 , 𝜎 1 , 𝜉 𝑛 ), since used -(𝑐, 𝜎 0 , 𝜉 𝑛 ) and 𝑏 𝜎 0 = true imply used -(𝑐 ′ , 𝜎 0 , 𝜉 𝑛 ).

Case 𝑐 ≡ (while 𝑏 {𝑐 ′ }). Consider the version of used -where the first parameter can be a state transformer 𝑓 : St → St ⊥ , instead of a command. Similarly, consider the version of the five properties claimed by the lemma where we use a state transformer 𝑓 : St → St ⊥ , again instead of a command. We denote both versions by used -(𝑓 , 𝜎 ′′ 0 , 𝜉 ′′ 𝑛 ) and 𝜑 (𝑓 , 𝜉 ′′ 𝑛 , 𝜎 ′′ 0 , 𝜎 ′′ 1 ). Let T be the subset of St → St ⊥ defined by

𝑓 ∈ T ⇐⇒ ∀𝜎 ′′ 0 , 𝜎 ′′ 1 ∈ St. ∀𝜉 ′′ 𝑛 ∈ St □ . used -(𝑓 , 𝜎 ′′ 0 , 𝜉 ′′ 𝑛 ) ∧ 𝜎 ′′ 0 ∼ 𝜉 ′′ 𝑛 𝜎 ′′ 1 =⇒ 𝜑 (𝑓 , 𝜉 ′′ 𝑛 , 𝜎 ′′ 0 , 𝜎 ′′ 1 ) ,
and 𝐹 :

[St → St ⊥ ] → [St → St ⊥ ]
be the following operator used in the semantics of the loop 𝑐 :

𝐹 (𝑓 ) (𝜎) ≜ if ( 𝑏 𝜎 = true) then 𝑓 † ( 𝑐 ′ 𝜎) else 𝜎.
We will show that T contains 𝜆𝜎. ⊥ and is closed under taking the least upper bound of an increasing chain in [St → St ⊥ ], and the operator 𝐹 preserves T . These three conditions imply that 𝑐 is in T , which in turn gives the five properties claimed by the lemma. The first condition holds simply because used -((𝜆𝜎.⊥), 𝜎 ′′ 0 , 𝜉 ′′ 𝑛 ) is false for all 𝜎 ′′ 0 and 𝜉 ′′ 𝑛 . To prove the closure under the least upper bound of a chain, consider an increasing sequence 𝑓 0 , 𝑓 1 , . ), as desired. It remains to show that 𝐹 (𝑓 ) ∈ T for all 𝑓 ∈ T . Pick 𝑓 ∈ T . We first replay our proof for the sequential-composition case after viewing 𝑓 † • 𝑐 ′ as the sequential composition of 𝑐 ′ and 𝑓 . This replay, then, gives the membership 𝑓 † • 𝑐 ′ ∈ T . Next, we replay our proof for the if case on 𝐹 (𝑓 ) after viewing 𝑓 † • 𝑐 ′ as the true branch and 𝜆𝜎. 𝜎 = skip as the false branch. This replay implies the required 𝐹 (𝑓 ) ∈ T . □

Lemma C.7. Let 𝑐 be a command, 𝜎, 𝜎 ′ ∈ St, and

𝜉 𝑛 ∈ St □ [Name]. • If 𝜎 | 𝑉 = 𝜎 ′ | 𝑉 for 𝑉 ≜ PVar ∪ dom(𝜉 𝑛 ) ∪ {like}, then used (𝑐, 𝜎, 𝜉 𝑛 ) implies used (𝑐, 𝜎 ′ , 𝜉 𝑛 ). • If 𝜎 | 𝑈 = 𝜎 ′ | 𝑈 for 𝑈 ≜ PVar ∪ dom(𝜉 𝑛 ), then used -(𝑐, 𝜎, 𝜉 𝑛 ) implies used -(𝑐, 𝜎 ′ , 𝜉 𝑛 ).
Proof. Assume the settings in the statement of this lemma. For the first claim, assume used (𝑐, 𝜎, 𝜉 𝑛 ). Then, by the definition of used and noerr,

𝑐 𝜎 ∈ St ∧ ∀𝜇 ∈ Name. 𝑐 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) ≤ 1 ∧ 𝜎 (like) = 1 ∧ 𝜉 𝑛 = 𝜎 | dom(𝜉 𝑛 ) ∧ dom(𝜉 𝑛 ) = {𝜇 ∈ Name | 𝑐 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 1} .
From this and Lemma C.6 (which is applicable since used (𝑐, 𝜎, 𝜉 𝑛 ) and 𝜎 | 𝑉 = 𝜎 ′ | 𝑉 ), we obtain In this proof, each equation involving integrals means (otherwise noted) that one side of the equation is defined if and only if the other side is defined, and when both sides are defined, they are the same.

𝑐 𝜎 ′ ∈ St ∧ ∀𝜇 ∈ Name. 𝑐 𝜎 ′ (cnt 𝜇 ) -𝜎 ′ (cnt 𝜇 ) ≤ 1 ∧ 𝜎 ′ (like) = 1 ∧ 𝜉 𝑛 = 𝜎 ′ | dom(𝜉 𝑛 ) ∧ dom(𝜉 𝑛 ) = {𝜇 ∈ Name | 𝑐 𝜎 ′ (cnt 𝜇 ) -𝜎 ′ (cnt 𝜇 ) = 1} .
Case 𝑐 ≡ skip, 𝑐 ≡ (𝑥 := 𝑒), or 𝑐 ≡ obs(𝑑, 𝑟 ). In this case, 𝑐 𝜋 ≡ 𝑐 so the desired equation holds.

Case 𝑐 ≡ (𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒)). If (𝑛, 𝑑, 𝜆𝑦.𝑒) ∉ dom(𝜋), then 𝑐 𝜋 ≡ 𝑐 and thus the desired equation holds. So assume that 𝜋 (𝑛, 𝑑, 𝜆𝑦.𝑒) = (𝑑 ′ , 𝜆𝑦 ′ .𝑒 ′ ) for some 𝑑 ′ and 𝜆𝑦 ′ .𝑒 ′ . Then, 𝑐 𝜋 ≡ (𝑥 := sam(𝑛, 𝑑 ′ , 𝜆𝑦 ′ .𝑒 ′ )).

First, by the validity of 𝜋, for all states 𝜎 ∈ St and measurable subsets

𝐴 ⊆ R, ∫ 1 [ 𝑒 [𝑟 /𝑦 ] 𝜎 ∈𝐴] • 𝑑 𝜎 (𝑟 ) 𝑑𝑟 = ∫ 1 [ 𝑒 ′ [𝑟 /𝑦 ′ ] 𝜎 ∈𝐴] • 𝑑 ′ 𝜎 (𝑟 ) 𝑑𝑟,
where both sides are always defined. Using this and the monotone convergence theorem, we can show that for all measurable 𝑓 : R → R, Using this and fv(𝑒), fv(𝑑) ⊆ PVar, we obtain the following: for any

∫ 𝑓 ( 𝑒 [𝑟 /𝑦] 𝜎) • 𝑑 𝜎 (𝑟 ) 𝑑𝑟 = ∫ 𝑓 ( 𝑒 ′ [𝑟 /𝑦 ′ ] 𝜎) • 𝑑 ′ 𝜎 (𝑟 ) 𝑑𝑟 . ( 17
𝜎 𝑟 ∈ St[Var \ PVar], 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (like) = 1, 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (pr 𝜇 ) = 𝑑 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (𝜎 𝑟 (𝜇)) = 𝑑 (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) (𝜎 𝑟 (𝜇)), 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (val 𝜇 ) = 𝑒 [𝜎 𝑟 (𝜇)/𝑦] (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = 𝑒 [𝜎 𝑟 (𝜇)/𝑦] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ), 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (cnt 𝜇 ) = 1, 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (cnt 𝜇 ′ ) = 0 for 𝜇 ′ 𝜇, 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 )| PVar = 𝜎 𝑝 [𝑥 ↦ → 𝑒 [𝜎 𝑟 (𝜇)/𝑦] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 )].
This implies that for any

𝜉 𝑛 ∈ St □ [Name], if prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ≠ 0, then dom(𝜉 𝑛 ) = {𝜇}, prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = 1 • 𝑑 (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) (𝜉 𝑛 (𝜇)), pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = 𝜎 𝑝 [𝑥 ↦ → 𝑒 [𝜉 𝑛 (𝜇)/𝑦] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 )], vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = [𝜇 ↦ → 𝑒 [𝜉 𝑛 (𝜇)/𝑦] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 )].
Note that the same equations hold for 𝑐 𝜋 , except that we replace 𝑑, 𝑒, and 𝑦 in the RHS of the above equations by 𝑑 ′ , 𝑒 ′ , and 𝑦 ′ . Using these, we obtain:

∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ [ {𝜇 }→R] 𝑑𝜉 𝑛 𝑑 (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) (𝜉 𝑛 (𝜇)) • 𝑔 𝑒 [𝜉 𝑛 (𝜇)/𝑦] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) = ∫ R 𝑑𝑟 𝑑 (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) (𝑟 ) • 𝑔 𝑒 [𝑟 /𝑦] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) = ∫ R 𝑑𝑟 𝑑 ′ (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) (𝑟 ) • 𝑔 𝑒 ′ [𝑟 /𝑦 ′ ] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) = ∫ [ {𝜇 }→R] 𝑑𝜉 𝑛 𝑑 ′ (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) (𝜉 𝑛 (𝜇)) • 𝑔 𝑒 ′ [𝜉 𝑛 (𝜇)/𝑦 ′ ] (𝜎 𝑝 ⊕ 𝜎 𝑟 0 ) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 )
where 𝑔 : R → R is defined as

𝑔(𝑟 ) = 𝑔(𝜎 𝑝 [𝑥 ↦ → 𝑟 ], [𝜇 ↦ → 𝑟 ]).
Here the first and fifth equalities use the equations proven above, the second and fourth equalities use that [{𝜇} → R] is isomorphic to R, and the third equality uses Eq. ( 17). This proves the desired equation.

Case 𝑐 ≡ (if 𝑏 {𝑐 ′ } else 𝑐 ′′ ). In this case, since fv(𝑏) ⊆ PVar and 𝑐 𝜋 ≡ if 𝑏 {𝑐 ′ 𝜋 } else 𝑐 ′′ 𝜋 , we have only two subcases:

• For all

𝜎 𝑟 ∈ St[Var \ PVar], 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = 𝑐 ′ (𝜎 𝑝 ⊕ 𝜎 𝑟 ) and 𝑐 𝜋 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = 𝑐 ′ 𝜋 (𝜎 𝑝 ⊕ 𝜎 𝑟 ). • For all 𝜎 𝑟 ∈ St[Var \ PVar], 𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = 𝑐 ′′ (𝜎 𝑝 ⊕ 𝜎 𝑟 ) and 𝑐 𝜋 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = 𝑐 ′′ 𝜋 (𝜎 𝑝 ⊕ 𝜎 𝑟 ).
If the first subcase holds, we have where the second equality is by IH on 𝑐 ′ . If the second subcase holds, we obtain a similar equation by IH on 𝑐 ′′ . Hence, the desired equation holds in all subcases.

∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ),
Case 𝑐 ≡ (𝑐 ′ ; 𝑐 ′′ ). In this case, we obtain the following equation:

∫ 𝑑𝜉 𝑛 prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 ′ 𝑛 prs □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 • 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 𝑔 pvars □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 , vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 = ∫ 𝑑𝜉 ′ 𝑛 prs □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • 𝑔 ′ pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 )
where

𝑔 ′ ( 𝜎 ′ 𝑝 , 𝜉 ′ 𝑛 ) ≜ ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) • 1 [dom( 𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 𝑔 pvars □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ⊕ vals □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) = ∫ 𝑑𝜉 ′ 𝑛 prs □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • 𝑔 ′ pvars □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), vals □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • • • ( * )
where the first equality is from Lemma C.9, the second equality uses dom(𝜉 ′ 𝑛 ) = dom(vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 )), and the third equality is by IH on 𝑐 ′ . We now analyse 𝑔 ′ ( 𝜎 ′ 𝑝 , 𝜉 ′ 𝑛 ) as follows:

𝑔 ′ ( 𝜎 ′ 𝑝 , 𝜉 ′ 𝑛 ) = ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) • 1 [dom( 𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 𝑔 pvars □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ⊕ vals □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) = ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) • 𝑔 ′′ pvars □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ), vals □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 )
where

𝑔 ′′ ( 𝜎 ′′ 𝑝 , 𝜉 ′′ 𝑛 ) ≜ 1 [dom( 𝜉 ′ 𝑛 )∩dom( 𝜉 ′′ 𝑛 )=∅] • 𝑔 𝜎 ′′ 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 = ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ 𝜋 ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) • 𝑔 ′′ pvars □ (𝑐 ′′ 𝜋 ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ), vals □ (𝑐 ′′ 𝜋 ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) = ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ 𝜋 ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) • 1 [dom( 𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 𝑔 pvars □ (𝑐 ′′ 𝜋 ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ⊕ vals □ (𝑐 ′′ 𝜋 ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 )
where the second and fourth equalities use dom(𝜉 ′′ 𝑛 ) = dom(vals □ (𝑐 ′′ ) ( 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 )), and the third equality is by IH on 𝑐 ′′ . Using this, we obtain the following equation for the main quantity ( * ):

( * ) = ∫ 𝑑𝜉 ′ 𝑛 prs □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ 𝜋 ) pvars □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 • 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 𝑔 pvars □ (𝑐 ′′ 𝜋 ) pvars □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 , vals □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐 ′′ 𝜋 ) pvars □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 = ∫ 𝑑𝜉 𝑛 prs □ (𝑐 ′ 𝜋 ; 𝑐 ′′ 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 ′ 𝜋 ; 𝑐 ′′ 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 ′ 𝜋 ; 𝑐 ′′ 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 )
where the first equality uses dom(𝜉 ′ 𝑛 ) = dom(vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 )), and the second equality is by Lemma C.9, as we did above. By 𝑐 ′ ; 𝑐 ′′ 𝜋 ≡ 𝑐 ′ 𝜋 ; 𝑐 ′′ 𝜋 , we get the desired equation.

Case 𝑐 ≡ (while 𝑏 {𝑐 ′ }). In this case, 𝑐 𝜋 ≡ (while 𝑏 {𝑐 ′ 𝜋 }). Without loss of generality, assume that 𝑔 is a nonnegative function; we can prove the general case of 𝑔 directly from the nonnegative case of 𝑔, by considering the nonnegative part and the negative part of 𝑔 separately.

Consider the version of prs □ (-), pvars □ (-), and vals □ (-), where the parameter can be a state transformer 𝑓 : St → St ⊥ , instead of a command. We denote the versions by prs □ (𝑓 ), pvars □ (𝑓 ), and

vals □ (𝑓 ). Define T ⊆ [St → St ⊥ ] 2 and 𝑇 : [St → St ⊥ ] 2 → [St → St ⊥ ] 2 by (𝑓 , 𝑓 ) ∈ T ⇐⇒ ∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ) (𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ) (𝜉 𝑛 ) for all measurable 𝑔 ′ : St[PVar] × St □ [Name] → R ≥0 and 𝜎 ′ 𝑝 ∈ St[PVar], 𝑇 (𝑓 , 𝑓 ) ≜ (𝐹 (𝑓 ), 𝐹 (𝑓 )),
where

𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ) ∈ St □ [Name] → R ≥0 and 𝐹, 𝐹 : [St → St ⊥ ] → [St → St ⊥ ] are defined by 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ) (𝜉 𝑛 ) ≜ prs □ (𝑓 ) (𝜎 ′ 𝑝 , 𝜉 𝑛 ) • 𝑔 ′ pvars □ (𝑓 ) (𝜎 ′ 𝑝 , 𝜉 𝑛 ), vals □ (𝑓 ) (𝜎 ′ 𝑝 , 𝜉 𝑛 ) , 𝐹 (𝑓 ) (𝜎) ≜ if ( 𝑏 𝜎 = true) then (𝑓 † • 𝑐 ′ ) (𝜎) else 𝜎, 𝐹 (𝑓 ) (𝜎) ≜ if ( 𝑏 𝜎 = true) then (𝑓 † • 𝑐 ′ 𝜋 ) (𝜎) else 𝜎.
Note that 𝐹 and 𝐹 are the operators used in the semantics of the loops 𝑐 and 𝑐 𝜋 , respectively. We will show that T contains (𝜆𝜎. ⊥, 𝜆𝜎. ⊥), the operator 𝑇 preserves T , and T is closed under taking the least upper bound of an increasing chain in [St → St ⊥ ] 2 , where the order on [St → St ⊥ ] 2 is defined as: (𝑓 0 , 𝑓 0 ) ⊑ (𝑓 1 , 𝑓 1 ) ⇐⇒ 𝑓 0 ⊑ 𝑓 1 ∧ 𝑓 0 ⊑ 𝑓 1 . These three conditions imply ( 𝑐 , 𝑐 𝜋 ) ∈ T , which in turn proves the desired equation:

∫ 𝑑𝜉 𝑛 prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 𝐺 𝑔,𝜎 𝑝 ( 𝑐 ) (𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 𝐺 𝑔,𝜎 𝑝 ( 𝑐 𝜋 ) (𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 prs □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) ,
where the second equality follows from ( 𝑐 , 𝑐 𝜋 ) ∈ T . The first condition holds simply because 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝜆𝜎. ⊥) (𝜉 𝑛 ) = 0 for all 𝑔 ′ , 𝜎 ′ 𝑝 , and 𝜉 𝑛 . To show the second condition, pick (𝑓 , 𝑓 ) ∈ T . Our goal is to show 𝑇 (𝑓 , 𝑓 ) ∈ T . We first replay our proof for the sequential-composition case on (𝑓 † • 𝑐 ′ , 𝑓 † • 𝑐 ′ 𝜋 ), after viewing 𝑓 † • 𝑐 ′ and 𝑓 † • 𝑐 ′ 𝜋 as the sequential composition of 𝑐 ′ and 𝑓 , and of 𝑐 ′ 𝜋 and 𝑓 , respectively. This replay, then, gives the membership (𝑓 † • 𝑐 ′ , 𝑓 † • 𝑐 ′ 𝜋 ) ∈ T . Next, we replay our proof for the if case on (𝐹 (𝑓 ), 𝐹 (𝑓 )), after viewing 𝑓 † • 𝑐 ′ and 𝑓 † • 𝑐 ′ 𝜋 as the true branches, and 𝜆𝜎. 𝜎 = skip as the false branch. This replay implies the required 𝑇 (𝑓 , 𝑓 ) = (𝐹 (𝑓 ), 𝐹 (𝑓 )) ∈ T .

To show the third condition, consider an increasing sequence

{(𝑓 𝑘 , 𝑓 𝑘 )} 𝑘 ∈N in T . Let 𝑓 ∞ ≜ 𝑘 ∈N 𝑓 𝑘 and 𝑓 ∞ ≜ 𝑘 ∈N 𝑓 𝑘 . Consider a measurable 𝑔 ′ : St[PVar] × St □ [Name] → R ≥0 and 𝜎 ′ 𝑝 ∈ St[PVar]. We should show that ∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ∞ ) (𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ∞ ) (𝜉 𝑛 ). Since {𝑓 𝑘 } 𝑘 ∈N is increasing, for any 𝜎 ∈ St, 𝑓 𝑘 (𝜎) ∈ St implies that 𝑓 𝑘 ′ (𝜎) = 𝑓 𝑘 (𝜎) ∈ St for all 𝑘 ′ ≥ 𝑘. Hence, {𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 𝑘 )} 𝑘 ∈N is a pointwise increasing sequence: for all 𝜉 𝑛 ∈ St □ [Name], 0 ≤ 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 𝑘 ) (𝜉 𝑛 ) ≤ 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 𝑘+1 ) (𝜉 𝑛 ) for all 𝑘 ∈ N.
Also, by the definition of 𝑓 ∞ , for any 𝜎 ∈ St, there exists 𝐾 ∈ N such that 𝑓 ∞ (𝜎) = 𝑓 𝐾 (𝜎); thus,

𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ∞ ) is the pointwise limit of {𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 𝑘 )} 𝑘 ∈N : for all 𝜉 𝑛 ∈ St □ [Name], 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ∞ ) (𝜉 𝑛 ) = lim 𝑘→∞ 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 𝑘 ) (𝜉 𝑛 ).
Note that the corresponding results hold for 𝑓 ∞ and 𝑓 𝑘 . Using these results, we finally obtain the following as desired:

∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ∞ ) (𝜉 𝑛 ) = lim 𝑘→∞ ∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 𝑘 ) (𝜉 𝑛 ) = lim 𝑘→∞ ∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 𝑘 ) (𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 𝐺 𝑔 ′ ,𝜎 ′ 𝑝 (𝑓 ∞ ) (𝜉 𝑛 ).
The first and third equalities follow from the monotone convergence theorem, applied to the above results. The second equality holds since (𝑓 𝑘 , 𝑓 𝑘 ) ∈ T . This completes the proof of the while case. □ Lemma C.8. Let 𝑐 be a command, 𝜎 0 , 𝜎 1 ∈ St, and 𝑟 0 ∈ R. Suppose that 𝜎 1 (like) = 𝜎 0 (like) • 𝑟 0 and

𝜎 1 | 𝑉 = 𝜎 0 | 𝑉 for 𝑉 ≜ Var \ {like}. If 𝑐 𝜎 0 ∈ St, then 𝑐 𝜎 1 ∈ St, 𝑐 𝜎 1 (like) = 𝑐 𝜎 0 (like) • 𝑟 0 , ( 𝑐 𝜎 1 )| 𝑉 = ( 𝑐 𝜎 0 )| 𝑉 .
Proof. Let 𝑉 ≜ Var \ {like}. Pick an arbitrary command 𝑐. We prove the lemma by induction on the structure of 𝑐. Let 𝜎 0 , 𝜎 1 ∈ St and 𝑟 0 ∈ R such that 𝑐 𝜎 0 ∈ St, 𝜎 1 (like) = 𝜎 0 (like) • 𝑟 0 , and

𝜎 1 | 𝑉 = 𝜎 0 | 𝑉 . We should show that 𝑐 𝜎 1 ∈ St, 𝑐 𝜎 1 (like) = 𝑐 𝜎 0 (like) • 𝑟 0 , and 𝑐 𝜎 1 | 𝑉 = 𝑐 𝜎 0 | 𝑉 .
Case 𝑐 ≡ skip. In this case, what we need to prove is identical to the assumption on (𝜎 0 , 𝜎 1 , 𝑟 0 ).

Case 𝑐 ≡ (𝑥 := 𝑒). By the semantics of the assignments, we have 𝑐 𝜎 1 ∈ St, and

𝑐 𝜎 1 (like) = 𝜎 1 (like) = 𝜎 0 (like) • 𝑟 0 = 𝑐 𝜎 0 (like) • 𝑟 0 .
The last requirement also holds since 𝑒 𝜎 0 = 𝑒 𝜎 1 and 𝜎 0 | 𝑉 = 𝜎 1 |𝑉 .

Case 𝑐 ≡ (𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒 ′ ). By the semantics of the sample commands, we have 𝑐 𝜎 1 ∈ St. Also, the assignments do not change the value of like, so that 𝑐 𝜎 1 (like

) = 𝜎 1 (like) = 𝜎 0 (like) • 𝑟 0 = 𝑐 𝜎 0 (like) • 𝑟 0 . It remains to show that 𝑐 𝜎 0 | 𝑉 = 𝑐 𝜎 1 | 𝑉 . Since 𝜎 0 | 𝑉 = 𝜎 1 | 𝑉 , we have 𝑛 𝜎 0 = 𝑛 𝜎 1 and 𝑑 𝜎 0 = 𝑑 𝜎 1 . Let 𝜇 ≜ 𝑛 𝜎 0 . Then, by the same reason, 𝑒 ′ [𝜇/𝑦] 𝜎 0 = 𝑒 ′ [𝜇/𝑦] 𝜎 1 . Let 𝑓 ≜ 𝑑 𝜎 0 and 𝑟 ≜ 𝑒 ′ [𝜇/𝑦] 𝜎 0 .
We prove the required equality as follows:

𝑐 𝜎 0 | 𝑉 = 𝜎 0 [𝑥 ↦ → 𝑟, val 𝜇 ↦ → 𝑟, pr 𝜇 ↦ → 𝑓 (𝜎 0 (𝜇)), cnt 𝜇 ↦ → 𝜎 0 (cnt 𝜇 ) + 1] | 𝑉 = 𝜎 1 [𝑥 ↦ → 𝑟, val 𝜇 ↦ → 𝑟, pr 𝜇 ↦ → 𝑓 (𝜎 0 (𝜇)), cnt 𝜇 ↦ → 𝜎 0 (cnt 𝜇 ) + 1] | 𝑉 = 𝜎 1 [𝑥 ↦ → 𝑟, val 𝜇 ↦ → 𝑟, pr 𝜇 ↦ → 𝑓 (𝜎 1 (𝜇)), cnt 𝜇 ↦ → 𝜎 1 (cnt 𝜇 ) + 1] | 𝑉 = 𝑐 𝜎 1 | 𝑉 .
Case 𝑐 ≡ (obs(𝑑, 𝑟 )). By the semantics of the observe commands, we have 𝑐 𝜎 1 ∈ St. Also, the observe commands do not change any variable except like. So,

𝑐 𝜎 0 | 𝑉 = 𝜎 0 |𝑉 = 𝜎 1 | 𝑉 = 𝑐 𝜎 1 | 𝑉 .
The remaining requirement for like can be proved as follows:

𝑐 𝜎 1 (like) = 𝜎 1 (like) • 𝑑 𝜎 1 (𝑟 ) = 𝜎 0 (like) • 𝑟 0 • 𝑑 𝜎 1 (𝑟 ) = 𝜎 0 (like) • 𝑟 0 • 𝑑 𝜎 0 (𝑟 ) = 𝑐 𝜎 0 (like) • 𝑟 0 .
Case 𝑐 ≡ (𝑐 ′ ; 𝑐 ′′ ). We have 𝑐 ′ 𝜎 0 ∈ St and 𝑐 ′′ ( 𝑐 ′ 𝜎 0 ) ∈ St. We apply induction hypothesis first to (𝑐 ′ , 𝜎 0 , 𝜎 1 , 𝑟 0 ), and then to (𝑐 ′′ , 𝑐 ′ 𝜎 0 , 𝑐 ′ 𝜎 1 , 𝑟 0 ). Then, we get the requirements of the lemma.

Case 𝑐 ≡ (if 𝑏 {𝑐 ′ } else {𝑐 ′′ }). We deal with the case that 𝑏 𝜎 0 = true. The other case of 𝑏 𝜎 0 = false can be proved similarly. Since 𝑏 𝜎 0 = true, we have 𝑐 ′ 𝜎 0 = 𝑐 𝜎 0 ∈ St. Thus, we can apply induction hypothesis to 𝑐 ′ . If we do so, we get

𝑐 ′ 𝜎 1 ∈ St, 𝑐 ′ 𝜎 1 (like) = 𝑐 ′ 𝜎 0 (like) • 𝑟 0 , and 𝑐 ′ 𝜎 1 | 𝑉 = 𝑐 ′ 𝜎 0 |𝑉 .
This gives the desired conclusion because 𝑏 𝜎 1 = 𝑏 𝜎 0 = true and so 𝑐 𝜎 1 = 𝑐 ′ 𝜎 1 , and 𝑐 𝜎 0 = 𝑐 ′ 𝜎 0 .

Case 𝑐 ≡ (while 𝑏 {𝑐 ′ }). Let 𝐹 be the operator on

[St → St ⊥ ] such that 𝑐 is the least fixed point of 𝐹 . Define a subset T of [St → St ⊥ ] as follows: a function 𝑓 ∈ [St → St ⊥ ] is in T if and only if for all 𝜎 ′ 0 , 𝜎 ′ 1 ∈ St such that 𝜎 ′ 1 | 𝑉 = 𝜎 ′ 0 | 𝑉 and 𝜎 ′ 1 (like) = 𝜎 ′ 0 (like) • 𝑟 0 , we have 𝑓 (𝜎 ′ 0 ) ≠ ⊥ =⇒ 𝑓 (𝜎 ′ 1 ) ≠ ⊥ ∧ 𝑓 (𝜎 ′ 1 )| 𝑉 = 𝑓 (𝜎 ′ 0 )| 𝑉 ∧ 𝑓 (𝜎 ′ 1 ) (like) = 𝑓 (𝜎 ′ 0 ) (like) • 𝑟 0 .
The set T contains the least function 𝜆𝜎.⊥, and is closed under the least upper bound of any chain in [St → St ⊥ ]. It is also closed under 𝐹 . This 𝐹 -closure follows essentially from our arguments for sequential composition, if command, and skip, and induction hypothesis on 𝑐 ′ . What we have shown for T implies that T contains the least fixed point of 𝐹 , which gives the desired property for 𝑐. □ Lemma C.9. Let 𝑐 ′ , 𝑐 ′′ be commands and 𝑔 :

St[PVar] × St □ [Name] → R be a measurable function. Then, for any 𝜎 𝑝 ∈ St[PVar], ∫ 𝑑𝜉 𝑛 prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 ′ 𝑛 prs □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • ∫ 𝑑𝜉 ′′ 𝑛 prs □ (𝑐 ′′ ) (pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ) • 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 𝑔 pvars □ (𝑐 ′′ ) (pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ), vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐 ′′ ) (pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ) . Proof. Let 𝑐 ′ , 𝑐 ′′ be commands, 𝑔 : St[PVar] × St □ [Name] → R
a measurable function, and 𝜎 𝑝 ∈ St [PVar]. In this proof, each equation involving integrals means (otherwise noted) that one side of the equation is defined if and only if the other side is defined, and when both sides are defined, they are the same.

First, to convert a single-integral on 𝜉 𝑛 to a double-integral on 𝜉 ′ 𝑛 and 𝜉 ′′ 𝑛 as in the desired equation, we show the following claim: for any measurable 𝑓 :

St □ [Name] → St and ℎ : St □ [Name] → R such that 𝑓 (𝜉 𝑛 )| dom(𝜉 𝑛 ) = 𝜉 𝑛 for all 𝜉 𝑛 ∈ St □ [Name], we have ∫ 𝑑𝜉 𝑛 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 ) ] • ℎ(𝜉 𝑛 ) = 𝐾 ⊆Name ∫ [𝐾→R] 𝑑𝜉 𝑛 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 ) ] • ℎ(𝜉 𝑛 ) = 𝐾 ⊆Name ∫ [𝐾→R] 𝑑𝜉 𝑛 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 ) ] • ℎ(𝜉 𝑛 ) • 𝐿 ⊆𝐾 1 [used (𝑐 ′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 | 𝐿 ) ] = 𝐾 ⊆Name 𝐿 ⊆𝐾 ∫ [𝐾→R] 𝑑𝜉 𝑛 1 [used (𝑐 ′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 | 𝐿 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 ) ] • ℎ(𝜉 𝑛 ) = 𝐾 ⊆Name 𝐿 ⊆𝐾 ∫ [𝐿→R] 𝑑𝜉 ′ 𝑛 ∫ [𝐾\𝐿→R] 𝑑𝜉 ′′ 𝑛 1 [used (𝑐 ′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ] • ℎ(𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = 𝐿 ′ ⊆Name 𝑀 ′ ⊆Name 𝐿 ′ ∩𝑀 ′ =∅ ∫ [𝐿 ′ →R] 𝑑𝜉 ′ 𝑛 ∫ [𝑀 ′ →R] 𝑑𝜉 ′′ 𝑛 1 [used (𝑐 ′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ] • ℎ(𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) Proc. ACM
= 𝐿 ′ ⊆Name ∫ [𝐿 ′ →R] 𝑑𝜉 ′ 𝑛 𝑀 ′ ⊆Name ∫ [𝑀 ′ →R] 𝑑𝜉 ′′ 𝑛 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 1 [used (𝑐 ′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ] • ℎ(𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = ∫ 𝑑𝜉 ′ 𝑛 ∫ 𝑑𝜉 ′′ 𝑛 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 1 [used (𝑐 ′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ] • ℎ(𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) .
The first equality uses the definition of St □ [Name] and its measure. The second equality uses that used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 𝑛 ) implies a unique existence of 𝐿 ⊆ dom(𝜉 𝑛 ) such that used (𝑐 ′ , 𝜎, 𝜉 𝑛 |𝐿); we already showed the existence of such 𝐿 in the proof of Lemma C.6 (for the sequential composition case), and the uniqueness follows from the definition of used. The third equality uses that 𝐾 is finite, and the fourth equality uses that

[𝐾 → R] is isomorphic to [𝐿 → R] × [𝐾 \ 𝐿 → R] for any 𝐿 ⊆ 𝐾. The fifth equality holds uses that {(𝐿, 𝐾 \ 𝐿) | 𝐾 ⊆ Name, 𝐿 ⊆ 𝐾 } = {(𝐿 ′ , 𝑀 ′ ) | 𝐿 ′ , 𝑀 ′ ⊆ Name, 𝐿 ′ ∩ 𝑀 ′ = ∅}.
The sixth equality uses that Name is finite, dom(𝜉 ′ 𝑛 ) = 𝐿 ′ , and dom(𝜉 ′′ 𝑛 ) = 𝑀 ′ . The seventh equality uses the definition of St □ [Name] and its measure.

Second, to decompose prs □ (𝑐 ′ ; 𝑐 ′′ ), pvars □ (𝑐 ′ ; 𝑐 ′′ ), and vals □ (𝑐 ′ ; 𝑐 ′′ ) as in the desired equation, we show the following claim. Suppose that 𝜎 ∈ St and

𝜉 ′ 𝑛 , 𝜉 ′′ 𝑛 ∈ St □ [Name] with dom(𝜉 ′ 𝑛 ) ∩dom(𝜉 ′′ 𝑛 ) = ∅ satisfy used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) and used (𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ). Then, we first get prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = 𝑐 ′ ; 𝑐 ′′ 𝜎 (like) • 𝜇 ∈dom(𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) 𝑐 ′ ; 𝑐 ′′ 𝜎 (pr 𝜇 ) = 𝑐 ′′ ( 𝑐 ′ 𝜎) (like) • 𝜇 ∈dom(𝜉 ′ 𝑛 ) 𝑐 ′′ ( 𝑐 ′ 𝜎) (pr 𝜇 ) • 𝜇 ∈dom(𝜉 ′′ 𝑛 ) 𝑐 ′′ ( 𝑐 ′ 𝜎) (pr 𝜇 ) = 𝑐 ′ 𝜎 (like) • 𝑐 ′′ ( 𝑐 ′ 𝜎) [like ↦ → 1] (like) • 𝜇 ∈dom(𝜉 ′ 𝑛 ) 𝑐 ′ 𝜎 (pr 𝜇 ) • 𝜇 ∈dom(𝜉 ′′ 𝑛 ) 𝑐 ′′ ( 𝑐 ′ 𝜎) [like ↦ → 1] (pr 𝜇 ) = prs □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ) • prs □ (𝑐 ′′ ) ( 𝑐 ′ 𝜎) [like ↦ → 1] | PVar , 𝜉 ′′ 𝑛 = prs □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ) • prs □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 .
Here is the proof of each equality.

• The first equality uses used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ). • The second equality uses noerr (𝑐 ′ ; 𝑐 ′′ , 𝜎), which comes from used (𝑐 ′ ; 𝑐 ′′ , 𝜎, -).

• The third equality comes from Lemma C.8, Lemma C.6-(2), and Lemma C.6-(3). The two applications of Lemma C.6 are valid since used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ) and dom(𝜉 ′ 𝑛 ) ∩ dom(𝜉 ′′ 𝑛 ) = ∅, where the first predicate follows from used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) and used (𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ) by the claim in the proof of Lemma C.6 (for the sequential composition case).

• The fourth equality uses that used (𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ) and used (𝑐 ′′ , (

𝑐 ′ 𝜎) [like ↦ → 1], 𝜉 ′′ 𝑛 )
, where the second predicate follows from used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ) and Lemma C.7.

• The fifth equality uses ( 𝑐 ′ 𝜎) [like ↦ → 1] | PVar = ( 𝑐 ′ 𝜎)| PVar = pvars □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 )
, where the second part of the equation comes from used (𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ). By the same argument so far (except that pr 𝜇 and × are replaced by val 𝜇 and ⊕), we next get

vals □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = vals □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 .
By a similar argument, we lastly get

pvars □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = 𝑐 ′ ; 𝑐 ′′ 𝜎 | PVar = 𝑐 ′′ ( 𝑐 ′ 𝜎)| PVar = 𝑐 ′′ ( 𝑐 ′ 𝜎) [like ↦ → 1] | PVar = pvars □ (𝑐 ′′ ) ( 𝑐 ′ 𝜎) [like ↦ → 1] | PVar , 𝜉 ′′ 𝑛 = pvars □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 .
Here is the proof of each equality.

• The first equality uses used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ). • The second equality uses noerr (𝑐 ′ ; 𝑐 ′′ , 𝜎) (shown above).

• The third equality uses used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ) (shown above) and Lemma C.6-(3). • The fourth equality uses used (𝑐 ′′ , (

𝑐 ′ 𝜎) [like ↦ → 1], 𝜉 ′′ 𝑛 ) (shown above). • The fifth equality uses ( 𝑐 ′ 𝜎) [like ↦ → 1] | PVar = pvars □ (𝑐 ′ ) (𝜎 | PVar , 𝜉 ′
𝑛 ) (shown above). Third, to remove some indicator terms that will appear in our derivation, we show the next claim: for any 𝜎 ∈ St and

𝜉 ′ 𝑛 , 𝜉 ′′ 𝑛 ∈ St □ [Name] with dom(𝜉 ′ 𝑛 ) ∩ dom(𝜉 ′′ 𝑛 ) = ∅, used -(𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ) and used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ) imply used -(𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 )
. Assume the premise. Then, we have

𝑐 ′ 𝜎 ∈ St ∧ 𝜉 ′ 𝑛 = 𝜎 | dom(𝜉 ′ 𝑛 ) ∧ ∀𝜇 ∈ Name. 𝑐 ′ 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) ≤ 1 ∧ dom(𝜉 ′ 𝑛 ) = {𝜇 ∈ Name | 𝑐 ′ 𝜎 ′ (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 1}, 𝑐 ′′ ( 𝑐 ′ 𝜎) ∈ St ∧ 𝜉 ′′ 𝑛 = ( 𝑐 ′ 𝜎)| dom(𝜉 ′′ 𝑛 ) ∧ ∀𝜇 ∈ Name. 𝑐 ′′ ( 𝑐 ′ 𝜎)(cnt 𝜇 ) -𝑐 ′ 𝜎 (cnt 𝜇 ) ≤ 1 ∧ dom(𝜉 ′′ 𝑛 ) = {𝜇 ∈ Name | 𝑐 ′′ ( 𝑐 ′ 𝜎) (cnt 𝜇 ) -𝑐 ′ 𝜎 (cnt 𝜇 ) = 1}. We should show 𝑐 ′ ; 𝑐 ′′ 𝜎 ∈ St ∧ 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 = 𝜎 | dom(𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ∧ ∀𝜇 ∈ Name. 𝑐 ′ ; 𝑐 ′′ 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) ≤ 1 ∧ dom(𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = {𝜇 ∈ Name | 𝑐 ′ ; 𝑐 ′′ 𝜎 ′ (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 1}.
We obtain the four clauses as follows. The first clause follows from 𝑐 ′′ ( 𝑐 ′ 𝜎) ∈ St. The second clause comes from

𝜉 ′ 𝑛 = 𝜎 | dom(𝜉 ′ 𝑛 ) and 𝜉 ′′ 𝑛 = ( 𝑐 ′ 𝜎)| dom(𝜉 ′′ 𝑛 ) = 𝜎 | dom(𝜉 ′′ 𝑛 )
, where the last equality comes from Lemma C.4. The third and fourth clauses hold by the following:

𝑐 ′ ; 𝑐 ′′ 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 𝑐 ′′ ( 𝑐 ′ 𝜎) (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) =          𝑐 ′ 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 1 if 𝜇 ∈ dom(𝜉 ′ 𝑛 ) 𝑐 ′′ ( 𝑐 ′ 𝜎) (cnt 𝜇 ) -𝑐 ′ 𝜎 (cnt 𝜇 ) = 1 if 𝜇 ∈ dom(𝜉 ′′ 𝑛 ) 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) = 0 if 𝜇 ∈ Name \ (dom(𝜉 ′ 𝑛 ) ∪ dom(𝜉 ′′ 𝑛 )). • The first case uses Lemma C.6-(2) (applied to used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ) and dom(𝜉 ′ 𝑛 ) ∩ dom(𝜉 ′′ 𝑛 ) = ∅)
∫ 𝑑𝜉 𝑛 prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) = ∫ 𝑑𝜉 𝑛 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 ) ] • prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ),
= ∫ 𝑑𝜉 ′ 𝑛 ∫ 𝑑𝜉 ′′ 𝑛 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 1 [used (𝑐 ′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ] • prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) • 𝑔 pvars □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), vals □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = ∫ 𝑑𝜉 ′ 𝑛 ∫ 𝑑𝜉 ′′ 𝑛 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • 1 [used (𝑐 ′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ] • prs □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • prs □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 • 𝑔 pvars □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 , vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 = ∫ 𝑑𝜉 ′ 𝑛 ∫ 𝑑𝜉 ′′ 𝑛 1 [dom(𝜉 ′ 𝑛 )∩dom(𝜉 ′′ 𝑛 )=∅] • prs □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • prs □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 • 𝑔 pvars □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 , vals □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 .
The first equality uses that prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) ≠ 0 implies 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 𝑛 ),𝜉 𝑛 ) ] = 1:

• Since prs □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 𝑛 ) ≠ 0, there is𝜎 𝑟 ∈ St[Var\(PVar∪dom(𝜉 𝑛 ))] such that used (𝑐 ′ ; 𝑐 ′′ , 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ). Note (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 )| 𝑉 = 𝑓 (𝜉 𝑛 )| 𝑉 for 𝑉 = PVar ∪ dom(𝜉 𝑛 ) ∪ {like}.
From these and Lemma C.7, we get used (𝑐 ′ ; 𝑐 ′′ , 𝑓 (𝜉 𝑛 ), 𝜉 𝑛 ). The second and third equalities use the first and second results we proved above, respectively. The fourth equality uses the next claim:

prs □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • prs □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ≠ 0 and dom(𝜉 ′ 𝑛 ) ∩ dom(𝜉 ′′ 𝑛 ) = ∅ imply 1 [used (𝑐 ′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ) ] • 1 [used (𝑐 ′ ;𝑐 ′′ ,𝑓 (𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ),𝜉 ′ 𝑛 ⊕𝜉 ′′ 𝑛 ) ] = 1.
We prove the claim using the third result we proved above:

• Assume the premise. From prs

□ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ≠ 0, there is 𝜎 ′ 𝑟 ∈ St[Var \ (PVar ∪ dom(𝜉 ′ 𝑛 ))] such that used (𝑐 ′ , 𝜎 𝑝 ⊕ 𝜉 ′ 𝑛 ⊕ 𝜎 ′ 𝑟 , 𝜉 ′ 𝑛 ). Note (𝜎 𝑝 ⊕ 𝜉 ′ 𝑛 ⊕ 𝜎 ′ 𝑟 )| 𝑉 ′ = 𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 )| 𝑉 ′ for 𝑉 ′ = PVar ∪ dom(𝜉 ′ 𝑛 ) ∪ {like}.
From these and Lemma C.7, we get used (𝑐 ′ , 𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ) as desired.

• From prs □ (𝑐 ′′ ) pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ≠ 0, there is 𝜎 ′′ 𝑟 ∈ St[Var\ (PVar∪dom(𝜉 ′′ 𝑛 ))] such that used (𝑐 ′′ , pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ 𝜉 ′′ 𝑛 ⊕ 𝜎 ′′ 𝑟 , 𝜉 ′′ 𝑛 ). Since used (𝑐 ′ , 𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ), we have 𝑐 ′ (𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 )) ∈ St and pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) = pvars □ (𝑐 ′ ) (𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 )| PVar , 𝜉 ′ 𝑛 ) = 𝑐 ′ (𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ))| PVar ; also, by Lemma C.4, 𝜉 ′′ 𝑛 = 𝑐 ′ (𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ))| dom(𝜉 ′′ 𝑛 ) . Thus, (pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ 𝜉 ′′ 𝑛 ⊕ 𝜎 ′′ 𝑟 )| 𝑉 ′′ = 𝑐 (𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ))| 𝑉 ′′ for 𝑉 ′′ = PVar ∪ dom(𝜉 ′′ 𝑛 ). From these and Lemma C.7, we get used -(𝑐 ′′ , 𝑐 ′ (𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 )), 𝜉 ′′ 𝑛 ). • From dom(𝜉 ′ 𝑛 ) ∩ dom(𝜉 ′′ 𝑛 ) = ∅, used -(𝑐 ′ , 𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ), and used -(𝑐 ′′ , 𝑐 ′ (𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 )), 𝜉 ′′ 𝑛 )
, we can apply the third result proved above, and get used Let 𝑐 be a command and 𝜋 be a simple reparameterisation plan. Suppose that for all 𝑛 ∈ NameEx, 𝑑, 𝑑 ′ ∈ DistEx, and (𝜆𝑦.𝑒) ∈ LamEx such that 𝜋 (𝑛, 𝑑, 𝜆𝑦.𝑒) = (𝑑 ′ , _), we have

-(𝑐 ′ ; 𝑐 ′′ , 𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ). Since 𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) (like) = 1, we get used (𝑐 ′ ; 𝑐 ′′ , 𝑓 (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 )
𝑑 ′ = dist N (𝑟 ′ 1 , 𝑟 ′ 2 ) for some 𝑟 ′ 1 , 𝑟 ′ 2 ∈ R. (18)
Further, assume that for all 𝜎 𝑛 ∈ St[Name], the function 19). Suppose that Eq. ( 20) does not hold. Then, 𝑓 is not a constant function.

𝜎 𝜃 ∈ St[𝜃 ] ↦ -→ 𝑝 ⟨rv (𝜋 ) ⟩ 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) (19)
On the one hand, since 𝑓 is continuous (by assumption) and not constant, the image of 𝑓 over its domain (i.e., 𝑓 (St[𝜃 ]) ⊆ R) is an uncountable set. This can be shown as follows: since the image of a connected set over a continuous function is connected,

𝑓 (St[𝜃 ]) is a connected set in R; since 𝑓 is not constant, 𝑓 (St[𝜃 ]) contains at least two points; since 𝑓 (St[𝜃 ]
) is connected, it should contain a non-empty interval, so it should be an uncountable set.

On the other hand, since 𝜋 is simple and satisfies Eq. ( 18), and since 𝑐 has only finitely many sample commands, 𝑓 (St[𝜃 ]) is a finite set. So this contradicts to that 𝑓 (St[𝜃 ]) is an uncountable set. Hence, 𝑓 should satisfy Eq. ( 20). □ Theorem D.2. Let 𝑓 : R × R 𝑛 → R be a measurable function that satisfies the next three conditions:

• For all 𝑥 ∈ R 𝑛 , 𝑓 (-, 𝑥) : R → R is differentiable. • For all 𝜃 ∈ R, ∫ R 𝑛 𝑓 (𝜃, 𝑥) 𝑑𝑥 is finite. • For all 𝜃 ∈ R, there is an open 𝑈 ⊆ R such that 𝜃 ∈ 𝑈 and ∫ R 𝑛 Lip 𝑓 (-, 𝑥)| 𝑈 𝑑𝑥 is finite. Here Lip(𝑔) for a function 𝑔 : 𝑉 → R with 𝑉 ⊆ R denotes the smallest Lipschitz constant of 𝑔: Lip(𝑔) ≜ sup 𝑟,𝑟 ′ ∈𝑉 , 𝑟 ≠𝑟 ′ |𝑔(𝑟 ′ ) -𝑔(𝑟 )| |𝑟 ′ -𝑟 | ∈ R ∪ {∞}.
Then, for all 𝜃 ∈ R, both sides of the following are well-defined and equal:

∇ 𝜃 ∫ R 𝑛 𝑓 (𝜃, 𝑥) 𝑑𝑥 = ∫ R 𝑛 ∇ 𝜃 𝑓 (𝜃, 𝑥) 𝑑𝑥
where ∇ 𝜃 denotes the partial differentiation operator with respect to 𝜃 .

Proof. This theorem follows from Theorem E.2 due to the following: the first condition of this theorem implies the first and second conditions of Theorem E.2 (as differentiability implies continuity); the second and third conditions of this theorem are identical to the third and fourth conditions of Theorem E.2; and the conclusion of this theorem is identical to that of Theorem E.2. □ D.2 Proof of Theorem 4.5

The proof of Theorem 4.5 relies on the following two lemmas, which are proven in §D.3. The first lemma states that if a command contains no observe commands, then its (full) density function can be decomposed into its partial density functions over 𝑆 and Name \ 𝑆 for any 𝑆 ⊆ Name. The second lemma states that if 𝜋 is simple and 𝑐 uses only 𝜆𝑦.𝑦 as the third argument of its sample commands, then the partial density function of 𝑐 𝜋 over non-transformed random variables (i.e., variables in Name \ rv(𝜋)) is connected to that of 𝑐 via the value function of 𝑐 𝜋 .

Lemma D.3. Let 𝑐 be a command. If 𝑐 does not contain observe commands, then, for all 𝑆 ⊆ Name,

𝜎 𝜃 ∈ St[𝜃 ], and 𝜎 𝑛 ∈ St[Name], 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ) = 𝑝 ⟨𝑆 ⟩ 𝑐,𝜎 𝜃 (𝜎 𝑛 ) • 𝑝 ⟨Name\𝑆 ⟩ 𝑐,𝜎 𝜃 (𝜎 𝑛 ).
Lemma D.4. Let 𝑐 be a command and 𝜋 be a reparameterisation plan. Suppose that every sample command in 𝑐 has 𝜆𝑦.𝑦 as its third argument. Then, for all We now prove Theorem 4.5 using the two lemmas.

𝜎 𝜃 ∈ St[𝜃 ] and 𝜎 𝑛 ∈ St[Name], if 𝑝 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) > 0, then 𝑝 ⟨Name\rv (𝜋 ) ⟩ 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) = 𝑝 ⟨Name\rv (𝜋 ) ⟩ 𝑐,𝜎 𝜃 (𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )).
Proof of Theorem 4.5. Let 𝑆 = rv(𝜋). Before starting the main derivation of the selective gradient estimator, we show the differentiability of several functions which are to be used in the derivation. From (R2) and (R3), the next functions over St[𝜃 ] are differentiable for all 𝜎 𝑛 by the preservation of differentiability under function composition:

𝜎 𝜃 ↦ -→ 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )), 𝜎 𝜃 ↦ -→ 𝑝 ⟨𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )), 𝜎 𝜃 ↦ -→ 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )), 𝜎 𝜃 ↦ -→ 𝑝 ⟨𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ), 𝜎 𝜃 ↦ -→ 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ).
From this, the next functions over St[𝜃 ] are also differentiable for all 𝜎 𝑛 by Lemma D.3 with 𝑐 𝑔 and 𝑐 𝑔 𝜋 and by the fact that the multiplication of differentiable functions is differentiable:

𝜎 𝜃 ↦ -→ 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )), 𝜎 𝜃 ↦ -→ 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ).
These differentiability results are required in the below proof to apply several gradients rules (e.g., ∇ 𝜃 (𝑓 (𝜃 ) + 𝑔(𝜃 )) = ∇ 𝜃 𝑓 (𝜃 ) + ∇ 𝜃 𝑔(𝜃 )) which may fail for non-differentiable functions.

Fix

𝜎 𝜃 ∈ St[𝜃 ].
Using the above differentiability results, we derive the selective gradient estimator as follows, where we write 𝜎 ′ 𝑛 for 𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ):

∇ 𝜃 L 𝜃 = ∇ 𝜃 ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 𝑛 ) = ∇ 𝜃 ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 ( 𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 ( 𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 ∇ 𝜃 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 ∇ 𝜃 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) + 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • ∇ 𝜃 log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) ∇ 𝜃 log 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) -∇ 𝜃 log 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) + ∇ 𝜃 log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) ∇ 𝜃 log 𝑝 ⟨𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) + ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) -∇ 𝜃 log 𝑝 ⟨𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) + ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) + ∇ 𝜃 log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) 0 + ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) -∇ 𝜃 log 𝑝 ⟨𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) + 0 + ∇ 𝜃 log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 ( 𝜎 ′ 𝑛 ) • log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) 𝑝 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) -∇ 𝜃 log 𝑝 ⟨𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) + ∇ 𝜃 log 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) .
We justify key steps of the above derivation below.

• The second equality comes from Theorem 4.2 and the fact that 𝑣 𝑐 𝑔 ,𝜎 𝜃 is the identity function (since the third argument of every sample command in 𝑐 𝑔 is the identity function 𝜆𝑦.𝑦).

• The third equality holds because differentiation there commutes with integration by (R5).

• The fourth comes from the product rule for differentiation: ∇ 𝜃 (𝑓 (𝜃 ) • 𝑔(𝜃 )) = ∇ 𝜃 𝑓 (𝜃 ) • 𝑔(𝜃 ) + 𝑓 (𝜃 ) •∇ 𝜃 𝑔(𝜃 ) for all differentiable 𝑓 and𝑔. Here 𝑓 and𝑔 in the original equation are differentiable because differentiability is preserved under division and log for positive-valued functions. • The fifth equality holds because ∇ 𝜃 𝑓 (𝜃 ) = 𝑓 (𝜃 ) • ∇ 𝜃 log 𝑓 (𝜃 ) for all differentiable and positivevalued 𝑓 . • The sixth equality follows from Lemma D.3 applied to 𝑐 𝑔 and 𝑐 𝑔 𝜋 (both of which do not contain observe commands), and from the linearity of differentiation: ∇ 𝜃 (𝑓 (𝜃 ) + 𝑔(𝜃 )) = ∇ 𝜃 𝑓 (𝜃 ) + ∇ 𝜃 𝑔(𝜃 ) for all differentiable 𝑓 and 𝑔. Here 𝑓 and 𝑔 in the original equation are differentiable because differentiability is preserved under log for positive-valued functions.

• The seventh equality follows from (R4) and

E 𝑝 𝑐𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = 0. ( 21
)
The proof of Eq. ( 21) will be given after we complete this justification of the derivation. • The last equality comes from Lemma D.4 applied to 𝑐 𝑔 .

The only remaining part is to prove Eq. ( 21). We derive the equation as follows:

∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 ′ 𝑛 ) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 ( 𝜎 𝑛 ) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • ∇ 𝜃 log 𝑝 ⟨Name\𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) + ∇ 𝜃 log 𝑝 ⟨𝑆 ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) = ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) • ∇ 𝜃 log 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) = ∫ 𝑑𝜎 𝑛 ∇ 𝜃 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) = ∇ 𝜃 ∫ 𝑑𝜎 𝑛 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) = ∇ 𝜃 1 = 0.
Here is the justification of the above derivation:

• The first equality comes from Lemma D.4 applied to 𝑐 𝑔 .

• The second equality follows from (R4).

• The third equality holds because of Lemma D.3 applied to 𝑐 𝑔 𝜋 (which does not contain observe commands), and the linearity of differentiation: ∇ 𝜃 (𝑓 (𝜃 ) + 𝑔(𝜃 )) = ∇ 𝜃 𝑓 (𝜃 ) + ∇ 𝜃 𝑔(𝜃 ) for all differentiable 𝑓 and 𝑔. Here 𝑓 and 𝑔 in the original equation are differentiable because differentiability is preserved under log for positive-valued functions.

• The fourth equality holds because ∇ 𝜃 𝑓 (𝜃 ) = 𝑓 (𝜃 ) • ∇ 𝜃 log 𝑓 (𝜃 ) for all differentiable and positive-valued 𝑓 . • The fifth equality uses (R5), which states the commutativity between differentiation and integration in the equality.

𝑝 ⟨𝑆 ⟩ 𝑐,𝜎 𝜃 (𝜎 𝑛 ) • 𝑝 ⟨Name\𝑆 ⟩ 𝑐,𝜎 𝜃 (𝜎 𝑛 ) = 𝑐 𝜎 (like) • 𝜇 ∈𝑆 𝑐 𝜎 (pr 𝜇 ) • 𝑐 𝜎 (like) • 𝜇 ∈Name\𝑆 𝑐 𝜎 (pr 𝜇 ) = 𝑐 𝜎 (like) 2 • 𝜇 ∈Name 𝑐 𝜎 (pr 𝜇 ) = 𝑐 𝜎 (like) • 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ) = 𝜎 (like) • 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ) = 𝑝 𝑐,𝜎 𝜃 (𝜎 𝑛 ).
The second last equality uses Lemma D.5, and the last equality uses 𝜎 (like) = 1 (which holds by the definition of 𝜎 0 ). This completes the proof. □

Proof of Lemma D.4. Consider a command 𝑐, a reparameterisation plan 𝜋, 𝜎 𝜃 ∈ St[𝜃 ], and 𝜎 𝑛 ∈ St [Name]. Assume that all the sample commands of 𝑐 have 𝜆𝑦.𝑦 as their third arguments, and 𝑝 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) > 0.

We first define several objects and make observations on them. Let 𝑆 ≜ Name \ rv(𝜋). Define 𝑓 * : St[Name] → St[AVar] to be the function for constructing an initial state: Proof. Let 𝑐 be a command that does not contain an observe command. We show the claim of the lemma by induction on the structure of 𝑐. Pick 𝜎 ∈ St such that 𝑐 𝜎 ∈ St. We will show that 𝑐 𝜎 (like) = 𝜎 (like).

𝑓 * (𝜎 𝑛 ) (𝑣) ≜              𝑓 pr (𝜎 𝑛 (𝜇)) if 𝑣 ≡
Case 𝑐 ≡ skip. In this case, 𝑐 𝜎 (like) = 𝜎 (like) by the definition of the semantics.

Case 𝑐 ≡ (𝑥 := 𝑒). Again, 𝑐 𝜎 (like) = 𝜎 (like) by the definition of the semantics.

Case𝑐 ≡ (𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒 ′ ). Once more, 𝑐 𝜎 (like) = 𝜎 (like) by the definition of the semantics. Case 𝑐 ≡ (𝑐 ′ ; 𝑐 ′′ ). We have 𝑐 ′ 𝜎 ∈ St and 𝑐 ′′ ( 𝑐 ′ 𝜎) ∈ St. We apply induction hypothesis first to (𝑐 ′ , 𝜎), and again to (𝑐 ′ , 𝑐 ′ 𝜎). The first application gives 𝑐 ′ 𝜎 (like) = 𝜎 (like), and the second 𝑐 ′ ; 𝑐 ′′ 𝜎 (like) = 𝑐 ′ 𝜎 (like). The desired conclusion follows from these two equalities.

Case 𝑐 ≡ (if 𝑏 {𝑐 ′ } else {𝑐 ′′ }). We deal with the case that 𝑏 𝜎 = true. The other case of 𝑏 𝜎 = false can be proved similarly. Since 𝑏 𝜎 = true, we have 𝑐 ′ 𝜎 = 𝑐 𝜎 ∈ St. Thus, we can apply induction hypothesis to 𝑐 ′ . If we do so, we get 𝑐 ′ 𝜎 (like) = 𝜎 (like). This gives the desired conclusion because 𝑐 𝜎 = 𝑐 ′ 𝜎.

Case 𝑐 ≡ (while 𝑏 {𝑐 ′ }). Let 𝐹 be the operator on [St → St ⊥ ] such that 𝑐 is the least fixed point of 𝐹 . Define a subset T of [St → St ⊥ ] as follows:

𝑓 ∈ T ⇐⇒ ∀𝜎 ′ ∈ St. 𝑓 (𝜎 ′ ) ∈ St =⇒ 𝑓 (𝜎 ′ ) (like) = 𝜎 ′ (like).
The set T contains the least function 𝜆𝜎.⊥, and is closed under the least upper bound of any chain in [St → St ⊥ ]. It is also closed under 𝐹 . This 𝐹 -closure follows essentially from our arguments for sequential composition, if command, and skip, and induction hypothesis on 𝑐 ′ . What we have shown for T implies that T contains the least fixed point of 𝐹 , which gives the desired property for 𝑐. □ Lemma D.6. Let 𝑐 be a command and 𝜋 be a reparameterisation plan. Suppose that every sample command in 𝑐 has 𝜆𝑦.𝑦 as its third argument. Then, for all 𝜎 𝑝 ∈ St [PVar] and

𝜉 𝑛 ∈ St □ [Name], if used (𝑐 𝜋 , 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) for some 𝜎 𝑟 , then ∃𝜎 𝑟 . used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ), pvars □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ),
𝑐 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (cnt 𝜇 ′ ) = (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (cnt 𝜇 ′ ) + 1 if 𝜇 ′ = 𝜇 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) (cnt 𝜇 ′ )
otherwise.

(

) 22
From this, we get dom(𝜉 𝑛 ) = dom(𝜉 𝑛 ) = {𝜇}. Further, by assumption, we get 𝑒 ≡ 𝑦. We now prove the three conclusions based on these observations and case analysis on (𝑛, 𝑑, 𝜆𝑦.𝑒).

First, assume (𝑛, 𝑑, 𝜆𝑦.𝑒) ∉ dom(𝜋). Then, 𝑐 𝜋 ≡ 𝑐 and

𝜉 𝑛 = vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = [𝜇 ↦ → 𝑐 𝜋 𝜎 (val 𝜇 )] = [𝜇 ↦ → 𝑒 [𝜎 (𝜇)/𝑦] 𝜎] = [𝜇 ↦ → 𝜉 𝑛 (𝜇)] = 𝜉 𝑛 ,
where the second last equality uses 𝑒 ≡ 𝑦. Hence, the three conclusions clearly hold.

Next, assume (𝑛, 𝑑, 𝜆𝑦.𝑒) ∈ dom(𝜋). Suppose that 𝜋 (𝑛, 𝑑, 𝜆𝑦.𝑒) = (𝑑, 𝜆𝑦.𝑒). Then, 𝑐 𝜋 ≡ (𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒)) and

𝜉 𝑛 = vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = [𝜇 ↦ → 𝑐 𝜋 𝜎 (val 𝜇 )] = [𝜇 ↦ → 𝑒 [𝜎 (𝜇)/𝑦] 𝜎] = [𝜇 ↦ → 𝑒 [𝜉 𝑛 (𝜇)/𝑦] 𝜎].
Since Eq. ( 22) holds also for 𝑐 𝜋 , and since dom(𝜉 𝑛 ) = {𝜇}, we get the first conclusion: used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ).

To prove the second conclusion, let 𝜎 = 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 . Then, for all 𝑣 ∈ PVar,

pvars □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) (𝑣) = 𝑐 𝜋 𝜎 (𝑣) = 𝑒 [𝜎 (𝜇)/𝑦] 𝜎 = 𝑒 [𝜉 𝑛 (𝜇)/𝑦] 𝜎 if 𝑣 ≡ 𝑥 𝜎 (𝑣) = 𝜎 𝑝 (𝑣) otherwise, pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) (𝑣) = 𝑐 𝜎 (𝑣) = 𝑒 [𝜎 (𝜇)/𝑦] 𝜎 = 𝜉 𝑛 (𝜇) = 𝑒 [𝜉 𝑛 (𝜇)/𝑦] 𝜎 if 𝑣 ≡ 𝑥 𝜎 (𝑣) = 𝜎 𝑝 (𝑣), otherwise,
where the second equation uses 𝑒 ≡ 𝑦. Hence, the second conclusion holds. For the third conclusion, let 𝑛 = name(𝛼, _). Then, 𝜇 = (𝛼, _) ∈ {(𝛼, 𝑖) ∈ Name | 𝑖 ∈ N} ⊆ rv(𝜋). Thus, dom(𝜉 𝑛 ) ∩ 𝑆 = dom(𝜉 𝑛 ) ∩ 𝑆 = {𝜇} ∩ 𝑆 = {𝜇} ∩ (Name \ rv(𝜋)) = ∅. From this, we get the third conclusion:

prs ⟨𝑆 ⟩ □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = 1 = prs ⟨𝑆 ⟩ □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ).
Case 𝑐 ≡ (𝑐 ′ ; 𝑐 ′′ ). First, we make several observations necessary to prove the conclusion. By used (𝑐 𝜋 , 𝜎, 𝜉 𝑛 ), we have

𝑐 ′ 𝜋 𝜎 ∈ St and 𝑐 ′′ 𝜋 ( 𝑐 ′ 𝜋 𝜎) ∈ St. Let 𝜎 ′ ≜ 𝑐 ′ 𝜋 𝜎, 𝜎 ′′ ≜ 𝑐 ′′ 𝜋 𝜎 ′ , 𝜎 ′ 𝑝 ≜ 𝜎 ′ | PVar , 𝜎 ′′ 𝑝 ≜ 𝜎 ′′ | PVar .
Then, by used (𝑐 𝜋 , 𝜎, 𝜉 𝑛 ) and the claim in the proof of Lemma C.6 (for the sequential composition case), there exist 𝜉 ′ 𝑛 and 𝜉 ′′ 𝑛 such that

𝜉 𝑛 = 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 , used (𝑐 ′ 𝜋 , 𝜎, 𝜉 ′ 𝑛 ), used -(𝑐 ′′ 𝜋 , 𝜎 ′ , 𝜉 ′′ 𝑛 ).
By the latter two, we can apply induction to (𝑐 ′ , 𝜎 𝑝 , 𝜉 ′ 𝑛 ) and (𝑐 ′′ , 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ), and IH gives the following:

pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) = pvars □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) [By IH on 𝑐 ′ ] = 𝑐 ′ 𝜋 (𝜎 [like ↦ → 1]) | PVar [By used (𝑐 ′ 𝜋 , 𝜎, 𝜉 ′ 𝑛 )] = ( 𝑐 ′ 𝜋 𝜎)| PVar = 𝜎 ′ | PVar = 𝜎 ′ 𝑝 , [By Lemma C.6-(3)] pvars □ (𝑐 ′′ ) (𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) = pvars □ (𝑐 ′′ 𝜋 ) (𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) [By IH on 𝑐 ′′ ] = 𝑐 ′′ 𝜋 (𝜎 ′ [like ↦ → 1]) | PVar [By used -(𝑐 ′′ 𝜋 , 𝜎 ′ , 𝜉 ′′ 𝑛 )] = ( 𝑐 ′′ 𝜋 𝜎 ′ )| PVar = 𝜎 ′′ | PVar = 𝜎 ′′ 𝑝 , [By Lemma C.6-(3)]
where

𝜉 ′ 𝑛 ≜ vals □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ≜ vals □ (𝑐 ′′ 𝜋 ) (𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 )
. By the former equation, we get Next, we show the second conclusion as follows:

𝜉 𝑛 = vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = vals □ (𝑐 ′ 𝜋 ; 𝑐 ′′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) [By 𝜉 𝑛 = 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ] = vals □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐 ′′ 𝜋 ) (pvars □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ) = vals □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) ⊕ vals □ (𝑐
pvars □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = ( 𝑐 𝜋 𝜎)| PVar [By used (𝑐 𝜋 , 𝜎, 𝜉 𝑛 )] = 𝑐 ′′ 𝜋 ( 𝑐 ′ 𝜋 𝜎) | PVar = 𝜎 ′′ | PVar = 𝜎 ′′ 𝑝 , pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) = pvars □ (𝑐 ′ ; 𝑐 ′′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) [By 𝜉 𝑛 = 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ] = pvars □ (𝑐 ′′ ) (pvars □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ) = pvars □ (𝑐 ′′ ) (𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ) = 𝜎 ′′ 𝑝 , [By the above]
where the second last equality uses used (𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ), used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), and the second claim in the proof of Lemma C.9.

Lastly, we show the third conclusion as follows:

prs ⟨𝑆 ⟩ □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = prs ⟨𝑆 ⟩ □ (𝑐 ′ ; 𝑐 ′′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) [By 𝜉 𝑛 = 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ] = prs ⟨𝑆 ⟩ □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • prs ⟨𝑆 ⟩ □ (𝑐 ′′ 𝜋 ) (pvars(𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ) = prs ⟨𝑆 ⟩ □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • prs ⟨𝑆 ⟩ □ (𝑐 ′′ 𝜋 ) (pvars(𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ) [By IH on 𝑐 ′ ] = prs ⟨𝑆 ⟩ □ (𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) • prs ⟨𝑆 ⟩ □ (𝑐 ′′ ) (pvars(𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ), 𝜉 ′′ 𝑛 ) [By IH on 𝑐 ′ and 𝑐 ′′ ] = prs ⟨𝑆 ⟩ □ (𝑐) (𝜎 𝑝 , 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) = prs ⟨𝑆 ⟩ □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ).
[By 𝜉 𝑛 = 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ] Here the second and fifth equalities use the second claim in the proof of Lemma C.9 with the following: used (𝑐 Proof. Note that the three functions + : R 2 → R, • : R 2 → R, and log : R >0 → R are all differentiable. Hence, applying Lemma E.4 produces the claim. □ Theorem E.2. Let 𝑓 : R × R 𝑛 → R be a measurable function that satisfies the next four conditions:

• For all 𝑥 ∈ R 𝑛 , 𝑓 (-, 𝑥) : R → R is continuous.

• For all 𝜃 ∈ R, ∇ 𝜃 𝑓 (𝜃, 𝑥) is well-defined for almost all 𝑥 ∈ R 𝑛 .

• For all 𝜃 ∈ R, ∫ R 𝑛 𝑓 (𝜃, 𝑥) 𝑑𝑥 is finite. • For all 𝜃 ∈ R, there is an open 𝑈 ⊆ R such that 𝜃 ∈ 𝑈 and ∫ R 𝑛 Lip 𝑓 (-, 𝑥)| 𝑈 𝑑𝑥 is finite. Here ∇ 𝜃 (-) and Lip(-) are defined as in Theorem D.2, and "almost all" is with respect to the Lebesgue measure. Then, for all 𝜃 ∈ R, both sides of the following are well-defined and equal:

∇ 𝜃 ∫ R 𝑛 𝑓 (𝜃, 𝑥) 𝑑𝑥 = ∫ R 𝑛 ∇ 𝜃 𝑓 (𝜃, 𝑥) 𝑑𝑥 .
Proof. Before starting the main proof, we show that for any open 𝑈 ′ ⊆ R, the following function 𝐿 : R 𝑛 → R ∪ {∞} is measurable:

𝐿(𝑥) ≜ Lip 𝑓 (-, 𝑥)| 𝑈 ′ . Define 𝑉 ⊆ R 2 and ℓ : 𝑉 × R 𝑛 → R as 𝑉 ≜ {𝑣 ∈ 𝑈 ′ × 𝑈 ′ | 𝑣 1 ≠ 𝑣 2 } ⊆ R 2 , ℓ (𝑣, 𝑥) ≜ |𝑓 (𝑣 1 , 𝑥) -𝑓 (𝑣 2 , 𝑥)| |𝑣 1 -𝑣 2 | .
Let 𝑉 ′ be a countable, dense subset of 𝑉 . Then, for all 𝑥 ∈ R 𝑛 ,

𝐿(𝑥) = sup 𝑣 ∈𝑉 ℓ (𝑣, 𝑥) = sup 𝑣 ′ ∈𝑉 ′ ℓ (𝑣 ′ , 𝑥),
where the first equality is by the definition of Lip(-), and the second equality holds since ℓ (-, 𝑥) : 𝑉 → R is continuous for all 𝑥 ∈ R 𝑛 by the first condition of this theorem. Since 𝐿 is the supremum of countably many measurable functions {ℓ (𝑣 ′ , -) : R 𝑛 → R | 𝑣 ′ ∈ 𝑉 ′ }, 𝐿 itself is a measurable function as desired. Note that the measurability of 𝐿 ensures that the integral ∫ R 𝑛 𝐿(𝑥) 𝑑𝑥 in the fourth condition of this theorem is well-defined (as a value in R ∪ {∞}).

We now start the main proof. Pick any 𝜃 ′ ∈ R. Define 𝑔 : R \ {𝜃 ′ } → R as

𝑔(𝜃 ) ≜ ∫ R 𝑛 𝑓 (𝜃, 𝑥) -𝑓 (𝜃 ′ , 𝑥) 𝜃 -𝜃 ′ 𝑑𝑥,
where 𝑔(𝜃 ) is finite by the third condition of this theorem. Then,

∇ 𝜃 ∫ R 𝑛 𝑓 (𝜃, 𝑥) 𝑑𝑥 𝜃 =𝜃 ′ = lim 𝜃 →𝜃 ′ 1 𝜃 -𝜃 ′ ∫ R 𝑛 𝑓 (𝜃, 𝑥) 𝑑𝑥 - ∫ R 𝑛 𝑓 (𝜃 ′ , 𝑥) 𝑑𝑥 = lim 𝜃 →𝜃 ′ 𝑔(𝜃 ),
where each equality denotes that LHS is well-defined if and only if RHS is well-defined, and if so, LHS and RHS are equal. Here the first equality is from the definition of ∇ 𝜃 , and the second equality from the third condition of this theorem. Note that the following are equivalent for any 𝑟 ∈ R:

(i) lim 𝜃 →𝜃 ′ 𝑔(𝜃 ) = 𝑟 . (ii) For any {𝜃 𝑖 } 𝑖 ∈N ⊆ R \ {𝜃 ′ }, lim 𝑖→∞ 𝜃 𝑖 = 𝜃 ′ implies lim 𝑖→∞ 𝑔(𝜃 𝑖 ) = 𝑟 .
So it suffices to show that (ii) holds with an appropriate choice of 𝑟 .

∫ R 𝑛 ℎ 𝑖 (𝑥) 𝑑𝑥 = ∫ R 𝑛 ∇ 𝜃 𝑓 (𝜃, 𝑥) 𝜃 =𝜃 ′ 𝑑𝑥,
where each equality denotes that both LHS and RHS are well-defined and are equal. Therefore, it suffices to show that the preconditions of the dominated convergence theorem for lim 𝑖→∞ ∫ R 𝑛 ℎ 𝑖 (𝑥) 𝑑𝑥 are satisfied.

We show that the preconditions indeed hold, which concludes the proof:

• "ℎ 𝑖 is measurable for all 𝑖 ∈ N": This holds by the measurability of 𝑓 .

• "lim 𝑖→∞ ℎ 𝑖 (𝑥) = (∇ 𝜃 𝑓 (𝜃, 𝑥))| 𝜃 =𝜃 ′ for almost all 𝑥 ∈ R 𝑛 ": This was already shown above.

(𝑥)| = |𝑓 (𝜃 𝑖 , 𝑥) -𝑓 (𝜃 ′ , 𝑥)| |𝜃 𝑖 -𝜃 ′ | ≤ Lip 𝑓 (-, 𝑥)| 𝑈 = 𝐻 (𝑥) for all 𝑥 ∈ R 𝑛 ,
where the inequality holds by the definition of Lip(-) with 𝜃 𝑖 , 𝜃 ′ ∈ 𝑈 and 𝜃 𝑖 ≠ 𝜃 ′ . □

Remark E.3. The second condition ( * ) of Theorem E.2 is weaker than the following, corresponding condition ( * ′ ) of a standard theorem for interchanging differentiation and integral (e.g., [Bogachev 2007, Corollary 2.8.7]): "for almost all 𝑥 ∈ R 𝑛 , ∇ 𝜃 𝑓 (𝜃, 𝑥) is well-defined for all 𝜃 ∈ R. " The difference arises from whether a proof uses the mean value theorem or not: a proof of the standard theorem finds the function 𝐻 for the dominated convergence theorem, by applying the mean value theorem which requires the stronger condition ( * ′ ); the proof of Theorem E.2 finds 𝐻 not by applying the mean value theorem (but by using the fourth condition of the theorem), so the weaker condition ( * ) is sufficient for the proof. In this sense, Theorem E.2 is close to [Bogachev 2007, Exercise 2.12.68]. □ Lemma E.4. Let 𝑓 : 𝑋 1 → 𝑋 2 and 𝑔 : 𝑋 2 → 𝑋 3 for some open sets 𝑋 𝑖 ⊆ R 𝑛 𝑖 . Suppose that 𝑓 is locally Lipschitz and 𝑔 is differentiable. Then, 𝑔 • 𝑓 : 𝑋 1 → 𝑋 3 is differentiable almost everywhere and the chain rule for 𝑔 • 𝑓 holds almost everywhere, i.e.,

𝐷 (𝑔 • 𝑓 ) (𝑥) = 𝐷 (𝑔) (𝑓 (𝑥)) • 𝐷 (𝑓 ) (𝑥)
for almost every 𝑥 ∈ 𝑋 1 . Here we use the Lebesgue measure as an underlying measure.

Proof. Since local Lipschitzness is preserved under a function composition, 𝑔 • 𝑓 is locally Lipschitz and thus differentiable almost everywhere. Since 𝑓 is also differentiable almost everywhere and 𝑔 is differentiable everywhere, the set , and pick 𝑣 ∈ Var. We have to show that 𝑝 (𝑣) ⊇ 𝑑 (𝑣) 𝑐 and 𝑑 (𝑣) ⊇ 𝑉 . We call these two requirements as conditions (i) and (ii).

𝑈 = 𝑋 1 \ {𝑥 ∈ 𝑋 1 | (𝑔 • 𝑓 is differentiable at 𝑥) ∧ (𝑔 is differentiable at 𝑓 (𝑥)) ∧ (𝑓 is differentiable at 𝑥)}
Case 𝑐 ≡ skip. In this case, 𝑝 (𝑣) = Var and 𝑉 = ∅, from which the conditions (i) and (ii) follow.

Case 𝑐 ≡ 𝑥 := 𝑒. In this case, 𝑉 = ∅. So, the condition (ii) holds. For the proof of the condition (i), we do case analysis on 𝑣. If 𝑣 is the updated variable 𝑥, we have 𝑝 (𝑣) = 𝑒 ♯ and 𝑑 (𝑣) = fv(𝑒). Since 𝑒 ♯ ⊇ fv(𝑒) 𝑐 , the condition holds. If 𝑣 is different from 𝑥, then 𝑝 (𝑣) is Var, and so it includes 𝑑 (𝑣) 𝑐 .

Case 𝑐 ≡ obs(dist N (𝑒 1 , 𝑒 2 ), 𝑟 ). The proof of this case is similar to the one for the assignments. Since 𝑉 = ∅, the condition (ii) holds. If 𝑣 is the variable like, then

𝑝 (𝑣) = like × pdf N (𝑟 ; 𝑒 1 , 𝑒 2 ) ♯ ⊇ fv(like × pdf N (𝑟 ; 𝑒 1 , 𝑒 2 )) 𝑐 = {like} ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 ) 𝑐 = 𝑑 (𝑣) 𝑐 .
So, the condition (i) holds in this case. If 𝑣 is not the variable like, then 𝑝 (𝑣) = Var, from which the condition (i) follows.

Case 𝑐 ≡ 𝑥 := sam(name(𝛼, 𝑒), dist N (𝑒 1 , 𝑒 2 ), 𝜆𝑦.𝑒 ′ ). In this case, 𝑉 = ∅, from which the condition (ii) follows. We do case analysis on whether 𝑒 is a real constant 𝑟 or not. During the case analysis, we use the assumption that fv(𝑒) 𝑐 ⊆ 𝑒 ♯ for all 𝑒, without mentioning it explicitly. First, we deal with the case that𝑒 ≡ 𝑟 . Let 𝜇 ≜ create_name(𝛼, 𝑟 ). If 𝑣 is none of 𝑥, val 𝜇 , pr 𝜇 , and cnt 𝜇 , we have 𝑝 (𝑣) = Var, which gives the condition (i). If 𝑣 ∈ {𝑥, val 𝜇 }, we prove the condition (i) as follows:

𝑑 (𝑣) 𝑐 = fv(𝑒 ′ [𝜇/𝑦]) 𝑐 ⊆ 𝑒 ′ [𝜇/𝑦] ♯ = 𝑝 (𝑣).
If 𝑣 ≡ pr 𝜇 , we calculate the condition (i) as follows:

𝑑 (pr 𝜇 ) 𝑐 = ({𝜇} ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 )) 𝑐 = fv(pdf N (𝜇; 𝑒 1 , 𝑒 2 )) 𝑐 ⊆ pdf N (𝜇; 𝑒 1 , 𝑒 2 ) ♯ = 𝑝 (pr 𝜇 ).
If 𝑣 ≡ cnt 𝜇 , we derive the condition (i) as follows:

𝑑 (cnt 𝜇 ) 𝑐 = {cnt 𝜇 } 𝑐 = fv(cnt 𝜇 + 1) 𝑐 ⊆ cnt 𝜇 + 1 ♯ = 𝑝 (cnt 𝜇 ).
Next, we handle the case that 𝑒 is not a real constant. If 𝑣 is none of 𝑥, val 𝜇 , pr 𝜇 , and cnt 𝜇 for some 𝜇 = (𝛼, _), we have 𝑝 (𝑣) = Var, which implies the condition (i). If 𝑣 ≡ 𝑥, we show the condition (i) as follows:

𝑝 (𝑣) = fv(𝑒) 𝑐 ∩ 𝜇=(𝛼,_) ∈Name 𝑒 ′ [𝜇/𝑦] ♯ ⊇ fv(𝑒) ∪ 𝜇=(𝛼,_) ∈Name fv(𝑒 ′ [𝜇/𝑦]) 𝑐 = 𝑑 (𝑣) 𝑐 .
If 𝑣 ≡ val 𝜇 for some 𝜇 = (𝛼, _), we calculate the condition (i) as follows:

𝑝 (𝑣) = fv(𝑒) 𝑐 ∩ 𝑒 ′ [𝜇/𝑦] ♯ ⊇ fv(𝑒) ∪ {val 𝜇 } ∪ fv(𝑒 ′ [𝜇/𝑦]) 𝑐 = 𝑑 (𝑣) 𝑐 .
If 𝑣 ≡ pr 𝜇 for some 𝜇 = (𝛼, _), we prove the condition (i) as follows:

𝑝 (𝑣) = fv(𝑒) 𝑐 ∩ pdf N (𝜇; 𝑒 1 , 𝑒 2 ) ♯ ⊇ fv(𝑒) 𝑐 ∩ fv(pdf N (𝜇; 𝑒 1 , 𝑒 2 )) 𝑐 ⊇ fv(𝑒) ∪ {𝜇, pr 𝜇 } ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 ) 𝑐 = 𝑝 (pr 𝜇 ).
Finally, if 𝑣 ≡ cnt 𝜇 for some 𝜇 = (𝛼, _), we show the condition (i) as follows:

𝑝 (𝑣) = fv(𝑒) 𝑐 ∩ cnt 𝜇 + 1 ♯ ⊇ fv(𝑒) 𝑐 ∩ fv(cnt 𝜇 + 1) 𝑐 = (fv(𝑒) ∪ {cnt 𝜇 }) 𝑐 = 𝑑 (𝑣) 𝑐 . Case 𝑐 ≡ 𝑐 ′ ; 𝑐 ′′ . Let (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯
and (𝑝 ′′ , 𝑑 ′′ , 𝑉 ′′ ) ≜ 𝑐 ′′ ♯ . The condition (ii) holds since

𝑑 (𝑣) = 𝑉 ′ ∪ 𝑑 ′ ∪ (𝑑 ′′ (𝑣)) ⊇ 𝑉 ′ ∪ 𝑑 ′ ∪ (𝑉 ′′ ) = 𝑉 .
For the condition (ii), we prove the required subset relationship as follows:

𝑑 (𝑣) 𝑐 = (𝑉 ′ ∪ 𝑑 ′ ∪ (𝑑 ′′ (𝑣))) 𝑐 = (𝑉 ′ ) 𝑐 ∩ 𝑤 ∈𝑑 ′′ (𝑣) 𝑑 ′ (𝑤) 𝑐 ⊆ (𝑉 ′ ) 𝑐 ∩ 𝑤 ∈𝑑 ′′ (𝑣) 𝑝 ′ (𝑤) ∩ 𝑤 ∈𝑝 ′′ (𝑣) 𝑐 𝑑 ′ (𝑤) 𝑐 = 𝑉 ′ ∪ 𝑝 ′ ∩ (𝑑 ′′ (𝑣)) 𝑐 ∪ 𝑑 ′ ∪ (𝑝 ′′ (𝑣) 𝑐 ) 𝑐 = 𝑝 (𝑣).
The subset relationship in the above derivation holds because 𝑑 ′ (𝑤) 𝑐 ⊆ 𝑝 ′ (𝑤) and 𝑑 ′′ (𝑣) ⊇ 𝑝 ′′ (𝑣) 𝑐 by induction hypothesis.

Case 𝑐 ≡ if 𝑏 {𝑐 ′ } else {𝑐 ′′ }. Let (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯ and (𝑝 ′′ , 𝑑 ′′ , 𝑉 ′′ ) ≜ 𝑐 ′′ ♯ . Then, by induction hypothesis, 𝑉 = fv(𝑏) ∪ 𝑉 ′ ∪ 𝑉 ′′ ⊆ fv(𝑏) ∪ 𝑑 ′ (𝑣) ∪ 𝑑 ′′ (𝑣) = 𝑑 (𝑣), which implies the condition (ii). Also, by induction hypothesis again,

𝑑 (𝑣) 𝑐 = fv(𝑏) 𝑐 ∩ 𝑑 ′ (𝑣) 𝑐 ∩ 𝑑 ′′ (𝑣) 𝑐 ⊆ fv(𝑏) 𝑐 ∩ 𝑝 ′ (𝑣) ∩ 𝑝 ′′ (𝑣) = 𝑝 (𝑣),
which shows the condition (ii).

Case 𝑐 ≡ while𝑏 {𝑐 ′ }. Let (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯
, and 𝐹 ♯ be the operator in the abstract semantics of 𝑐. Note that the abstract domain D ♯ contains (𝑝 ⊥ , 𝑑 ⊥ , 𝑉 ⊥ ) = ((𝜆𝑣.Var), (𝜆𝑣.∅), ∅). Thus, it is sufficient to show that 𝐹 ♯ is a well-defined monotone function on D ♯ , because then the least fixed point of 𝐹 ♯ is also in D ♯ and satisfies the conditions (i) and (ii). The monotonicity of 𝐹 ♯ holds because when (𝑝 1 , 𝑑 1 , 𝑉 1 ) ≜ 𝐹 ♯ (𝑝 0 , 𝑑 0 , 𝑉 0 ), the inputs 𝑝 0 , 𝑑 0 , and 𝑉 0 are used in the right polarity in the definitions of 𝑝 1 , 𝑑 1 , and 𝑉 1 ; for instance, 𝑝 0 is used only in the positive position (with respect to the subset order) when it is used to define 𝑝 1 . To prove well-definedness of 𝐹 ♯ , assume that 𝑝 0 (𝑣 0 ) ⊇ 𝑑 0 (𝑣 0 ) 𝑐 and 𝑑 0 (𝑣 0 ) ⊇ 𝑉 0 for all 𝑣 0 ∈ Var, and pick a variable 𝑣 1 ∈ Var. Then, since 𝑉 0 ⊆ 𝑑 0 (𝑣 1 ),

𝑉 1 = fv(𝑏) ∪ 𝑑 ′ ∪ (𝑉 0 ) ∪ 𝑉 ′ ⊆ fv(𝑏) ∪ 𝑑 ′ ∪ (𝑑 0 (𝑣 1 )) ∪ 𝑉 ′ ∪ {𝑣 1 } = 𝑑 1 (𝑣 1 ).
Also, by the induction hypothesis on the loop body 𝑐 ′ and the relationship 𝑑 0 (𝑣 1 ) ⊇ 𝑝 0 (𝑣 1 ) 𝑐 ,

𝑑 1 (𝑣 1 ) 𝑐 = fv(𝑏) 𝑐 ∩ (𝑉 ′ ) 𝑐 ∩ 𝑤 ∈𝑑 0 (𝑣 1 ) 𝑑 ′ (𝑤) 𝑐 ∩ {𝑣 1 } 𝑐
Case 𝑐 ≡ (𝑥 := sam(𝑛, dist N (𝑒 1 , 𝑒 2 ), 𝜆𝑦.𝑒 ′ )). The sample commands always terminate. So, the condition (i) holds. We prove the condition (ii) by case analysis on 𝑛.

The first case is that𝑛 is a constant expression, i.e., it is an expression of the form name(𝛼, 𝑟 ) for some 𝛼 ∈ Str and real number𝑟 . Let 𝜇 ≜ create_name(𝛼, 𝑟 ). If𝑣 is not one of𝑥, val 𝜇 , and pr 𝜇 , then𝑑 (𝑣) = {𝑣 }, and 𝑐 𝜎 (𝑣) = 𝑔(𝜎 (𝑣)) and 𝑐 𝜎 ′ (𝑣) = 𝑔(𝜎 ′ (𝑣)) for some function 𝑔 : R → R, so that the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ implies that 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣), as desired. If 𝑣 is 𝑥 or val 𝜇 , then 𝑑 (𝑣) = fv(𝑒 ′ [𝜇/𝑦]), 𝑐 𝜎 (𝑣) = 𝑒 ′ [𝜇/𝑦] 𝜎, and 𝑐 𝜎 ′ (𝑣) = 𝑒 ′ [𝜇/𝑦] 𝜎 ′ , so that the required 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣) holds. Finally, if 𝑣 = pr 𝜇 , then 𝑑 (𝑣) = {𝜇} ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 ), and so, the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ implies that

𝑐 𝜎 (pr 𝜇 ) = pdf N (𝜇; 𝑒 1 , 𝑒 2 ) 𝜎 = pdf N (𝜇; 𝑒 1 , 𝑒 2 ) 𝜎 ′ = 𝑐 𝜎 ′ (pr 𝜇 ),
which is precisely the equality that we want. The next case is that 𝑛 is not a constant expression. Let name(𝛼, 𝑒) be the form of 𝑛. If 𝑣 is not one of 𝑥, val 𝜇 , pr 𝜇 , and cnt 𝜇 for some 𝜇 of the form (𝛼, _), then 𝑑 (𝑣) = {𝑣 }, 𝑐 𝜎 (𝑣) = 𝜎 (𝑣), and 𝑐 𝜎 ′ (𝑣) = 𝜎 ′ (𝑣), so that the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ implies the desired 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣). Assume that 𝑣 is one of 𝑥, val 𝜇 , pr 𝜇 , and cnt 𝜇 for some 𝜇 with 𝜇 = (𝛼, _). Let 𝜇 0 ≜ 𝑛 𝜎 and 𝜇 ′ 0 ≜ 𝑛 𝜎 ′ . Since 𝑑 (𝑣) ⊇ fv(𝑛) in this case, the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ ensures that 𝜇 0 = 𝜇 ′ 0 . If 𝑣 is 𝑥, then fv(𝑒 ′ [𝜇 0 /𝑦]) ⊆ 𝑑 (𝑣), so that the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ gives the desired

𝑐 𝜎 (𝑣) = 𝑒 ′ [𝜇 0 /𝑦] 𝜎 = 𝑒 ′ [𝜇 ′ 0 /𝑦] 𝜎 ′ = 𝑐 𝜎 ′ (𝑣). If 𝑣 is cnt 𝜇 for some 𝜇 of the form (𝛼, _), then cnt 𝜇 ∈ 𝑑 (𝑣), and 𝑐 𝜎 (cnt 𝜇 ) = 𝑔(𝜎 (cnt 𝜇 )) and 𝑐 𝜎 ′ (cnt 𝜇 ) = 𝑔(𝜎 ′ (cnt 𝜇 )) for some function 𝑔 : R → R, so that 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣), as desired. If 𝑣 is val 𝜇 for 𝜇 = (𝛼, _), then 𝑑 (𝑣) ⊇ {val 𝜇 } ∪ fv(𝑒 ′ [𝜇/𝑦]), and 𝑐 𝜎 (val 𝜇 ) = ℎ( 𝑒 ′ [𝜇/𝑦] 𝜎, 𝜎 (val 𝜇 )) and 𝑐 𝜎 ′ (val 𝜇 ) = ℎ( 𝑒 ′ [𝜇/𝑦] 𝜎 ′ , 𝜎 ′ (val 𝜇 )) for some ℎ : R × R → R, so that 𝑐 𝜎 (val 𝜇 ) = 𝑐 𝜎 ′ (val 𝜇 ) as desired.
Finally, if 𝑣 is pr 𝜇 for some 𝜇 of the form (𝛼, _), then 𝑑 (𝑣) ⊇ {pr 𝜇 , 𝜇} ∪ fv(𝑒 1 ) ∪ fv(𝑒 2 ), and for some 𝑘 : R 4 → R, 𝑐 𝜎 (𝑣) = 𝑘 (𝜎 (pr 𝜇 ), 𝜎 (𝜇), 𝑒 1 𝜎, 𝑒 2 𝜎) and 𝑐 𝜎 ′ (𝑣) = 𝑘 (𝜎 ′ (pr 𝜇 ), 𝜎 ′ (𝜇), 𝑒 1 𝜎 ′ , 𝑒 2 𝜎 ′ ), so that the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ guarantees that 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣), as desired.

Case 𝑐 ≡ (𝑐 ′ ; 𝑐 ′′ ). Let (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯ and (𝑝 ′′ , 𝑑 ′′ , 𝑉 ′′ ) ≜ 𝑐 ′′ ♯ . Recall that

𝑑 (𝑣) = 𝑉 ′ ∪ (𝑑 ′ ) ∪ (𝑑 ′′ (𝑣)) = 𝑉 ′ ∪ {𝑑 ′ (𝑣 ′′ ) | 𝑣 ′′ ∈ 𝑑 ′′ (𝑣)} and 𝑉 = 𝑉 ′ ∪ (𝑑 ′ ) ∪ (𝑉 ′′ ).
Let us handle the condition (i) first. Since 𝑐 ′ ; 𝑐 ′′ 𝜎 0 ∈ St, we have 𝑐 ′ 𝜎 0 ∈ St. But 𝜎 0 ∼ 𝑉 ′ 𝜎 ′ 0 , because 𝜎 0 and 𝜎 ′ 0 are ∼ 𝑉 -related and 𝑉 includes 𝑉 ′ . Thus, 𝑐 ′ 𝜎 ′ 0 ∈ St as well by induction hypothesis, and it is sufficient to show 𝑐 ′ 𝜎 0 ∼ 𝑉 ′′ 𝑐 ′ 𝜎 ′ 0 . Note that for every 𝑣 ′′ ∈ 𝑉 ′′ , by the definition of 𝑉 , we have 𝑉 ⊇ 𝑑 ′ (𝑣 ′′ ), and so 𝜎 0 ∼ 𝑑 ′ (𝑣 ′′ ) 𝜎 ′ 0 , which implies, by induction hypothesis, that 𝑐 ′ 𝜎 0 ∼ {𝑣 ′′ } 𝑐 ′ 𝜎 ′ 0 . As a result, we have the desired 𝑐 ′ 𝜎 0 ∼ 𝑉 ′′ 𝑐 ′ 𝜎 ′ 0 . Next, we deal with the condition (ii). Since 𝑐 ′ ; 𝑐 ′′ 𝜎 and 𝑐 ′ ; 𝑐 ′′ 𝜎 ′ are both in St, there exist states 𝜎 1 , 𝜎 ′ 1 such that 𝑐 ′ 𝜎 = 𝜎 1 and 𝑐 ′ 𝜎 ′ = 𝜎 ′ 1 . We apply the induction hypothesis to 𝑐 ′ and get 𝜎 1 ∼ 𝑑 ′′ (𝑣) 𝜎 ′ 1 . Since 𝑐 ′′ 𝜎 1 and 𝑐 ′′ 𝜎 ′ 1 are in St, we apply the induction hypothesis again but this time to 𝑐 ′′ , 𝜎 1 , and 𝜎 ′ 1 , and obtain

𝑐 ′′ 𝜎 1 ∼ {𝑣 } 𝑐 ′′ 𝜎 ′ 1 , which implies the desired 𝑐 𝜎 (𝑣) = 𝑐 ′′ 𝜎 1 (𝑣) = 𝑐 ′′ 𝜎 ′ 1 (𝑣) = 𝑐 𝜎 ′ (𝑣). Case 𝑐 ≡ (if 𝑏 {𝑐 ′ } else {𝑐 ′′ }). Let (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯ and (𝑝 ′′ , 𝑑 ′′ , 𝑉 ′′ ) ≜ 𝑐 ′′ ♯ . Then, 𝑑 (𝑣) = fv(𝑏) ∪ 𝑑 ′ (𝑣) ∪ 𝑑 ′′ (𝑣) and 𝑉 = fv(𝑏) ∪ 𝑉 ′ ∪ 𝑉 ′′ .
We prove the condition (𝑖) under the assumption that 𝑏 𝜎 0 = true. Essentially the same proof applies to the other case that 𝑏 𝜎 0 = false. Since 𝑉 includes fv(𝑏), we also have 𝑏 𝜎 ′ 0 = true. Furthermore, since 𝑉 ′ ⊆ 𝑉 and so 𝜎 0 ∼ 𝑉 ′ 𝜎 ′ 0 by the induction hypothesis, we get that 𝑐 ′ 𝜎 ′ 0 ∈ St.

(d) |= Δ( 𝑐 ′′ , 𝑑 ′′ (𝑣), {𝑣 }). We obtain (b) as follows: by induction hypothesis on 𝑐 ′′ , we have |= Φ( 𝑐 ′′ , 𝑝 ′′ (𝑣), {𝑣 }), and by the weakening lemma for Φ (Lemma F.4), we have (b). We obtain (d) directly by Theorem F.1 on 𝑐 ′′ . For (a), consider induction hypothesis on 𝑐 ′ , which says that |= Φ( 𝑐 ′ , 𝑝 ′ (𝑤), {𝑤 }) for all 𝑤 ∈ Var. By the merging lemma for Φ (Lemma F.6), we have |= Φ( 𝑐 ′ , (𝑝 ′ ) ∩ (𝑑 ′′ (𝑣) ∩ 𝑝 ′′ (𝑣)), 𝑑 ′′ (𝑣) ∩ 𝑝 ′′ (𝑣)). Since 𝑝 (𝑣) ⊆ (𝑝 ′ ) ∩ (𝑑 ′′ (𝑣)) ⊆ (𝑝 ′ ) ∩ (𝑑 ′′ (𝑣) ∩ 𝑝 ′′ (𝑣)), we obtain (a) by the weakening lemma for Φ (Lemma F.4). For (c), observe that

𝑝 (𝑣) 𝑐 ⊇ 𝑉 ′ ∪ (𝑑 ′ ) ∪ (𝑝 ′′ (𝑣) 𝑐 ) and 𝑑 ′′ (𝑣) \ (𝑑 ′′ (𝑣) ∩ 𝑝 ′′ (𝑣)) = 𝑝 ′′ (𝑣) 𝑐 (23)
where the second equality follows from 𝑝 ′′ (𝑣) ⊇ 𝑑 ′′ (𝑣) (c) 𝑝 (𝑣) 𝑐 ⊇ fv(𝑏). We obtain (a) and (b) as follows: by induction hypothesis on 𝑐 ′ and 𝑐 ′′ , we have |= Φ( 𝑐 ′ , 𝑝 ′ (𝑣), {𝑣 }) and |= Φ( 𝑐 ′′ , 𝑝 ′′ (𝑣), {𝑣 }), and by Eq. ( 24) and the weakening lemma for Φ (Lemma F.4), we have (a) and (b). We obtain (c) directly by Eq. ( 24).

Case 𝑐 ≡ (while 𝑏 {𝑐 0 }). The proof starts by decomposing 𝑐 and 𝑐 ♯ into smaller pieces. Let

(𝑝 0 , 𝑑 0 , 𝑉 0 ) ≜ 𝑐 0 ♯ . Define 𝐹 : [St → St ⊥ ] → [St → St ⊥ ]
and 𝐹 ♯ : D ♯ → D ♯ as in §3 and Fig. 3:

𝐹 (𝑡) (𝜎) ≜ 𝜎 if 𝑏 𝜎 = false 𝑡 † ( 𝑐 0 𝜎) if 𝑏 𝜎 = true, 𝐹 ♯ (𝑝, 𝑑, 𝑉 ) ≜ 𝜆𝑣. fv(𝑏) 𝑐 ∩ (𝑉 0 ∪ (𝑝 0 ) ∩ (𝑑 (𝑣)) 𝑐 ∪ (𝑑 0 ) ∪ (𝑝 (𝑣) 𝑐 )) 𝑐 , 𝜆𝑣. fv(𝑏) ∪ 𝑉 0 ∪ (𝑑 0 ) ∪ (𝑑 (𝑣)) ∪ {𝑣 }, fv(𝑏) ∪ (𝑑 0 ) ∪ (𝑉 ) ∪ 𝑉 0 . Define 𝑡 ′ 𝑛 ∈ [St → St ⊥ ] and (𝑝 ′ 𝑛 , 𝑑 ′ 𝑛 , 𝑉 ′ 𝑛 ) ∈ D ♯ for 𝑛 ∈ N ∪ {∞} as 𝑡 ′ 𝑛 ≜ 𝜆𝜎. 𝐹 𝑛 (𝑡 ⊥ ) (𝜎) if 𝑛 ∈ N 𝑖 ∈N 𝑡 ′ 𝑖 if 𝑛 = ∞, (𝑝 ′ 𝑛 , 𝑑 ′ 𝑛 , 𝑉 ′ 𝑛 ) ≜ (𝐹 ♯ ) 𝑛 (𝑝 ⊥ , 𝑑 ⊥ , 𝑉 ⊥ ), if 𝑛 ∈ N 𝑖 ∈N (𝑝 ′ 𝑖 , 𝑑 ′ 𝑖 , 𝑉 ′ 𝑖 ) if 𝑛 = ∞,
where 𝑡 ⊥ = 𝜆𝜎. ⊥ and (𝑝 ⊥ , 𝑑 ⊥ , 𝑉 ⊥ ) = (𝜆𝑣. Var, 𝜆𝑣 . ∅, ∅). Then, we have

𝑐 = 𝑡 ′ ∞ and 𝑐 ♯ = (𝑝 ′ ∞ , 𝑑 ′ ∞ , 𝑉 ′ ∞ ). The proof is organized as follows. Define 𝑇 ,𝑇 ′ ⊆ [St → St ⊥ ] as 𝑇 ≜ {𝑓 ∈ [St → St ⊥ ] | ∀𝑣 ∈ Var. |= Δ(𝑓 , 𝑑 ′ ∞ (𝑣), {𝑣 }) ∧ |= Δ(𝑓 , 𝑉 ′ ∞ , ∅)}, 𝑇 ′ ≜ {𝑓 ∈ [St → St ⊥ ] | ∀𝑣 ∈ Var. |= Φ(𝑓 , 𝑝 ′ ∞ (𝑣), {𝑣 })}.
In Theorem F.1, we proved

𝑡 ′ 𝑛 ∈ 𝑇 for all 𝑛 ∈ N ∪ {∞}. (25)
In this theorem, our goal is to show 𝑡 ′ ∞ ∈ 𝑇 ′ . To do so, we prove the next three statements:

(a) 𝑡 ′ 0 ∈ 𝑇 ′ . (b) If 𝑡 ′ ∈ 𝑇 ′ ∩ 𝑇 , then 𝐹 (𝑡 ′ ) ∈ 𝑇 ′ . (c) If 𝑡 ′ 𝑛 ∈ 𝑇 ′ for all 𝑛 ∈ N, then 𝑡 ′ ∞ ∈ 𝑇 ′ .
It suffices to prove the three because (a), (b), and Eq. ( 25) imply 𝑡 ′ 𝑛 ∈ 𝑇 ′ for all 𝑛 ∈ N, and this and (c) imply 𝑡 ′ ∞ ∈ 𝑇 ′ . We now prove (a), (b), and (c) as follows. First, (a) follows directly from Lemma F.9. Next, we prove (b). Consider 𝑡 ′ ∈ 𝑇 ′ ∩𝑇 . Our goal is to show |= Φ(𝐹 (𝑡 ′ ), 𝑝 ′ ∞ (𝑣), {𝑣 }) for all 𝑣 ∈ Var. Observe that 𝐹 (𝑡 ′ ) = cond ( 𝑏 , seq( 𝑐 0 , 𝑡 ′ ), skip ).

By Theorem F.1 and induction hypothesis on 𝑐 0 , we have 𝑐 0 ∈ 𝛾 (𝑝 0 , 𝑑 0 , 𝑉 0 ), and by assumption, we have

𝑡 ′ ∈ 𝛾 (𝑝 ′ ∞ , 𝑑 ′ ∞ , 𝑉 ′ ∞ ) = 𝑇 ′ ∩ 𝑇 .
By applying to these the proofs of skip, sequential composition, and conditional cases, we have |= Φ(𝐹 (𝑡 ′ ), 𝑝 ′′ (𝑣), {𝑣 }) for all 𝑣 ∈ Var where

𝑝 ′′ (𝑣) = fv(𝑏) 𝑐 ∩ 𝑉 0 ∪ (𝑝 0 ) ∩ (𝑑 ′ ∞ (𝑣)) 𝑐 ∪ (𝑑 0 ) ∪ (𝑝 ′ ∞ (𝑣) 𝑐 ) 𝑐 ∩ Var. Since 𝑝 ′′ is the 𝑝 part of 𝐹 ♯ (𝑝 ′ ∞ , 𝑑 ′ ∞ , 𝑉 ′ ∞ ) and (𝑝 ′ ∞ , 𝑑 ′ ∞ , 𝑉 ′ ∞ )
is a fixed point of 𝐹 ♯ , we have 𝑝 ′′ = 𝑝 ′ ∞ . Hence, we obtain |= Φ(𝐹 (𝑡 ′ ), 𝑝 ′ ∞ (𝑣), {𝑣 }) for all 𝑣 ∈ Var. This completes the proof of (b). Finally, we prove (c). Suppose that 𝑡 ′ 𝑛 ∈ 𝑇 ′ for all 𝑛 ∈ N, and let 𝑣 ∈ Var. Our goal is to show |= Φ(𝑡 ′ ∞ , 𝑝 ′ ∞ (𝑣), {𝑣 }).

Observe that Lemma F.10 implies the goal when applied to 𝐾 = 𝑝 ′ ∞ (𝑣), 𝐿 = {𝑣 }, and {𝑓 𝑛 } 𝑛 ∈N = {𝑡 ′ 𝑛 } 𝑛 ∈N . Hence, it suffices to show the three preconditions of the lemma: (ii) For all 𝑣 ∈ Var,

• {𝑡 ′ 𝑛 } 𝑛
𝑝 ′ ∞ (𝑣) 𝑐 ⊇ 𝑉 ′ ∞ .
We give the proof of the two statements below. This completes the proof of the while-loop case. Proof of (ii). We prove a stronger statement: for all 𝑛 ∈ N and 𝑣 ∈ Var, 𝑝 ′ 𝑛 (𝑣) 𝑐 ⊇ 𝑉 ′ 𝑛 . This statement implies (ii) because 𝑝 ′ ∞ (𝑣) 𝑐 = ( 𝑛 ∈N 𝑝 ′ 𝑛 (𝑣)) 𝑐 = 𝑛 ∈N 𝑝 ′ 𝑛 (𝑣) 𝑐 ⊇ 𝑛 ∈N 𝑉 ′ 𝑛 = 𝑉 ′ ∞ . We prove the statement by induction on 𝑛. For 𝑛 = 0, we have 𝑝 ′ 𝑛 (𝑣) 𝑐 = Var 𝑐 = ∅ ⊇ ∅ = 𝑉 ′ 𝑛 for all 𝑣 ∈ Var. For 𝑛 > 0, let 𝑣 ∈ Var. By induction hypothesis, 𝑝 ′ 𝑛-1 (𝑣) 𝑐 ⊇ 𝑉 ′ 𝑛-1 holds. Using this, we have 𝑝 ′ 𝑛 (𝑣) 𝑐 = fv(𝑏) ∪ 𝑉 0 ∪ (𝑝 0 ) ∩ (𝑑 ′ 𝑛-1 (𝑣)) 𝑐 ∪ (𝑑 0 ) ∪ (𝑝 ′ 𝑛-1 (𝑣) 𝑐 ) ⊇ fv(𝑏) ∪ 𝑉 0 ∪ (𝑑 0 ) ∪ (𝑉 ′ 𝑛-1 ) = 𝑉 ′ 𝑛 . This completes the proof of (ii).

Proof of (i). This completes the proof of (i).

Proof of (iii). We prove this by induction on 𝑛. For 𝑛 = 0, 𝑡 ′ 𝑛 (𝜎) = ⊥ and 𝐼 𝑛 (𝜎) = ∅ for all 𝜎 ∈ St. Hence, Eq. ( 26 Given these, it suffices to show the claim that ℎ = ℎ 2 • ℎ 1 for some 𝜏 1 and 𝜏 2 : if the claim holds, then we have ℎ = ℎ 2 • ℎ 1 ∈ 𝜙 𝐾,𝑀 by Assumption 6, since ℎ 1 ∈ 𝜙 𝐾,𝐿 and ℎ 2 ∈ 𝜙 𝐿,𝑀 . We prove the claim by case analysis on 𝑓 (-⊕ 𝜏). Then, {𝑔 𝑛 } 𝑛 ∈N is an 𝜔-chain when we order 𝑔 𝑛 's by graph inclusion, and 𝑔 ∞ is the least upper bound of this chain for the same order. We will show that 𝑔 ∞ = 𝑔 𝑛 for some 𝑛 ∈ N by case analysis on Proof of Theorem 5.10. We go through the assumptions, and show that they are satisfied by 𝜙 (𝑑) and 𝜙 (𝑙) .

Σ
Case of Assumption 3. Let 𝐾, 𝐿 ⊆ Var such that 𝐿 ⊆ 𝐾. The projection 𝜋 𝐾,𝐿 is total and has an open set as its domain. Furthermore, the projection 𝜋 𝐾,𝐿 is differentiable and 1-Lipschitz continuous. Since Lipschitz continuity implies local Lipschitz continuity, we have both 𝜋 𝐾,𝐿 ∈ 𝜙 (𝑑) 𝐾,𝐿 and 𝜋 𝐾,𝐿 ∈ 𝜙 (𝑙) 𝐾,𝐿 , as desired. We should show that 𝑓 and 𝑔 satisfy 𝜙 (𝑑) 𝐾,𝐿 and 𝜙 (𝑙) 𝐾,𝐿 , respectively. In both cases, dom(𝑓 ) and dom(𝑔) are the intersections of two open sets, so that they are open as required.

To prove the differentiability of 𝑓 , consider 𝜎 ∈ dom(𝑓 ). Let ℎ 0 and ℎ 1 be the linear functions in + lim

𝜎 ′ →0 ∥𝑓 1 (𝜎 + 𝜎 ′ ) -𝑓 1 (𝜎) -ℎ 1 (𝜎 ′ ) ∥ 2 ∥𝜎 ′ ∥ 2 2 = 0.
Thus, 𝑓 is differentiable at 𝜎, as desired. It remains to prove the local Lipschitzness of 𝑔. Pick 𝜎 ∈ dom(𝑔). Then, 𝑔 0 and 𝑔 1 are defined at 𝜎 and they are locally Lipschitz. Thus, there exist open sets 𝑂 0 ⊆ dom(𝑔 0 ) and 𝑂 1 ⊆ dom(𝑔 1 ) and constants 𝐵 0 , 𝐵 1 > 0 such that 𝜎 belongs to both 𝑂 0 and 𝑂 1 , and for all 𝜎 0 , 𝜎 ′ 0 ∈ 𝑂 0 and 𝜎 1 , 𝜎 ′ 1 ∈ 𝑂 1 , ∥𝑔 0 (𝜎 0 ) -𝑔 0 (𝜎 ′ 0 ) ∥ 2 ≤ 𝐵 0 ∥𝜎 0 -𝜎 ′ 0 ∥ 2 and ∥𝑔 1 (𝜎 1 ) -𝑔 1 (𝜎 ′ 1 ) ∥ 2 ≤ 𝐵 1 ∥𝜎 1 -𝜎 ′ 1 ∥ 2 . Let 𝑂 ≜ 𝑂 0 ∩ 𝑂 1 . The set 𝑂 is open, and contains 𝜎. Furthermore, for all 𝜎 ′ , 𝜎 ′′ ∈ 𝑂, ∥𝑔(𝜎 ′ ) -𝑔(𝜎 ′′ ) ∥ 2 = ∥𝑔 0 (𝜎 ′ ) -𝑔 0 (𝜎 ′′ ) ∥ 2 2 + ∥𝑔 1 (𝜎 ′ ) -𝑔 1 (𝜎 ′′ ) ∥ 2 2 ≤ 𝐵 2 0 ∥𝜎 ′ -𝜎 ′′ ∥ 2 2 + 𝐵 2 1 ∥𝜎 ′ -𝜎 ′′ ∥ 2 2 = 𝐵 2 0 + 𝐵 2 1 • ∥𝜎 ′ -𝜎 ′′ ∥ 2 . Thus, 𝑔 is Lipschitz in 𝑂, as desired. Thus, 𝑔 1 is Lipschitz in 𝑂 ′ , as desired.

Case of Assumption 6. For the composition condition, we handle the differentiability case only. The other case can be proved similarly. Consider Assume that 𝑓 ∈ 𝜙 (𝑑) 𝐾,𝐿 and 𝑔 ∈ 𝜙 (𝑑) 𝐿,𝑀 . Let ℎ be the standard composition of partial functions 𝑔 and 𝑓 . We should show that ℎ ∈ 𝜙 (𝑑) 𝐾,𝑀 as well, that is, dom(ℎ) is open and ℎ is differentiable on its domain. Note that dom(ℎ) = dom(𝑓 ) ∩ 𝑓 -1 (dom(𝑔)).

Since 𝑔 ∈ 𝜙 (𝑑) 𝐿,𝑀 , the set dom(𝑔) is open. Because 𝑓 ∈ 𝜙 (𝑑) 𝐾,𝐿 , dom(𝑓 ) is open and 𝑓 is continuous on its domain. The latter implies that 𝑓 -1 (dom(𝑔)) is open as well. Thus, the intersection of dom(𝑓 ) and 𝑓 -1 (dom(𝑔)) is open as desired. The differentiability of ℎ on its domain holds since the restriction of 𝑓 to dom(ℎ) gives a differentiable total function from dom(ℎ) to dom(𝑔), that of 𝑔 to dom(𝑔) is also a differentiable total function, and the composition of two differentiable functions is differentiable.

Case of Assumption 7. The empty set is open, and the empty function is jointly differentiable and locally Lipschitz continuous. Thus, the strictness assumption holds for both predicate families. □ Third part: We show that 𝜋 satisfies (R3), i.e., the functions in (R3) are differentiable with respect to 𝜃 jointly. For this, it suffices to show the claim that for all 𝑣 ∈ {pr 𝜇 , val 𝜇 | 𝜇 ∈ Name} and 𝜎 𝑛 ∈ St[Name], the following function is differentiable with respect to 𝜃 jointly:

𝜎 𝜃 ∈ St[𝜃 ] ↦ -→ 𝑐 𝑔 𝜋 (𝜎 𝑝\𝜃 ⊕ 𝜎 𝜃 ⊕ 𝜎 𝑛 ⊕ 𝑔(𝜎 𝑛 )) (𝑣). (31)
This implies (R3) because:𝑐 𝑔 𝜋 does not have a double-sampling error, so each function in (R2) is either a multiplication or a pairing of the function in Eq. ( 31) (for different𝑣); and multiplication and pairing preserve differentiability. To show the claim, consider any 𝑣 ∈ {pr 𝜇 , val 𝜇 | 𝜇 ∈ Name}. Then, 𝜃 ⊆ p 𝑔 (𝑣) by Eq. ( 13). By the soundness of differentiability analysis (Theorem 5.8), |= Φ( 𝑐 𝑔 𝜋 , p 𝑔 (𝑣), {𝑣 }). From this, and by the weakening lemma of Φ with 𝜃 ⊆ p 𝑔 (𝑣) (Lemma F.4), we have |= Φ( 𝑐 𝑔 𝜋 , 𝜃, {𝑣 }). Hence, the function in Eq. ( 31) is differentiable with respect to 𝜃 jointly. This completes the overall proof.

□ Lemma H.1. Let 𝑓 : R 𝑛 × R 𝑚 → R be a function such that

𝑓 (𝑥, 𝑦) = 𝑓 1 (𝑥) if 𝑥 ∈ 𝑈 𝑓 2 (𝑦) if 𝑥 ∉ 𝑈
for some 𝑓 1 : R 𝑛 → R, 𝑓 2 : R 𝑚 → R, and 𝑈 ⊆ 𝑅 𝑛 . Suppose that 𝑓 2 (R 𝑚 ) = R and 𝑓 (-, 𝑦) : R 𝑛 → R is continuous for all 𝑦 ∈ R 𝑚 . Then, 𝑈 is either ∅ or R 𝑛 .

Proof. Here is a sketch of the proof. We prove the lemma by contradiction. Suppose that 𝑈 is neither ∅ nor R 𝑛 . Then, the boundary of 𝑈 (i.e., bd(𝑈 ) ⊆ R 𝑛 ) is nonempty, since the boundary of a set is empty if and only if the set is both open and closed, and since ∅ and R 𝑛 are the only subsets of R 𝑛 that are both open and closed. Let 𝑥 ∈ bd(𝑈 ) and consider two cases: 𝑥 ∈ 𝑈 or 𝑥 ∉ 𝑈 . In each of the two cases, we can show that there exists 𝑦 ∈ R 𝑚 such that 𝑓 (-, 𝑦) is not continuous at 𝑥. When showing the discontinuity, we use the following: 𝑥 ∈ bd(𝑈 ); the specific way that 𝑓 is defined (in terms of 𝑈 , 𝑓 1 , and 𝑓 2 ); and the assumption that 𝑓 2 (R 𝑚 ) = R. By assumption, 𝑓 (-, 𝑦) should be continuous over the entire R 𝑛 , so we get contradiction. □

I DEFERRED RESULTS IN §7

I.1 Deferred Experiment Details and Results

Table 7. Pyro examples used in experiments and their key features (continued from Table 3). The last five columns show the total number of code lines (excluding comments), loops, sample commands, observe commands, and learnable parameters (declared explicitly by pyro.param or implicitly by a neural network module). Each number is the sum of the counts in the model and guide.

Name

  then 𝑐 𝜎 else 𝑐 ′ 𝜎, while 𝑏 {𝑐} 𝜎 ≜ (fix 𝐹 ) (𝜎) where 𝐹 (𝑓 ) (𝜎) ≜ if ( 𝑏 𝜎 = true) then 𝑓 † ( 𝑐 𝜎) else 𝜎, 𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒 ′ ) 𝜎 ≜ 𝜎 [𝑥 ↦ → 𝑟, val 𝜇 ↦ → 𝑟, pr 𝜇 ↦ → 𝑑 𝜎 (𝜎 (𝜇)), cnt 𝜇 ↦ → 𝜎 (cnt 𝜇 ) + 1] where 𝜇 ≜ 𝑛 𝜎 and 𝑟 ≜ 𝑒 ′ [𝜇/𝑦] 𝜎, obs(𝑑, 𝑟 ) 𝜎 ≜ 𝜎 [like ↦ → 𝜎 (like) • 𝑑 𝜎 (𝑟 )].

Fig. 3 .

 3 Fig. 3. Abstract semantics of commands defining 𝑐 ♯ ∈ D ♯ .

  in the interior of its domain}, 𝜙 (pd) 𝐾,𝐿 ≜ {𝑓 : St[𝐾] ⇀ St[𝐿] | dom(𝑓 ) is open, and for all 𝑣 ∈ 𝐾, 𝑓 is partially differentiable in 𝑣 }, 𝜙 (pc) 𝐾,𝐿 ≜ {𝑓 : St[𝐾] ⇀ St[𝐿] | dom(𝑓 ) is open, and for all 𝑣 ∈ 𝐾, 𝑓 is partially continuous in 𝑣 }.

  ) using the following 𝜎 𝑝 ∈ St[PVar] and 𝑓 * : St[Name] → St[AVar]:

  used (𝑐, 𝜎, 𝜉 𝑛 ) and used (𝑐, 𝜎 ′ , 𝜉 𝑛 ), we have 𝜎 (like) = 1 = 𝜎 ′ (like) and so 𝜎 | 𝑉 = 𝜎 ′ | 𝑉 for 𝑉 = PVar ∪ dom(𝜉 𝑛 ) ∪ {like}. Using this and used (𝑐, 𝜎, 𝜉 𝑛 ), we can apply Lemma C.6-(3) and -(4) to get 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣) for all 𝑣 ∈ PVar ∪ {like} ∪ {pr 𝜇 , val 𝜇 | 𝜇 ∈ dom(𝜉 𝑛 )}. Hence, we obtain the desired equations in Eq. (16). □ Proof of Lemma C.2. Let 𝑐 be a command, ℎ : St[Name] → R be a measurable function, 𝑓 * : St[Name] → St[AVar] be the function defined in the statement of this lemma, and 𝜎 𝑝 ∈ St[PVar].

  Note that we have the first clause by Lemma C.6-(1), the second and fifth clauses by Lemma C.6-(5), and the third and fourth clauses by 𝜎 | 𝑉 = 𝜎 ′ | 𝑉 . Hence, used (𝑐, 𝜎 ′ , 𝜉 𝑛 ) holds. The proof of the second claim is exactly the same except that we apply Lemma C.6 to 𝜎 | 𝑈 = 𝜎 ′ | 𝑈 to prove only the four clauses of used that exclude 𝜎 ′ (like) = 1. □ C.3 Proof of Lemma C.3 Proof of Lemma C.3. We prove this lemma by induction on the structure of 𝑐. Let 𝑔 : St[PVar] × St □ [Name] → R be a measurable function and 𝜎 𝑝 ∈ St[PVar].

  ) Next, choose any 𝜎 𝑟 0 ∈ St[Var \ PVar]. Since fv(𝑛) ⊆ PVar, there exists 𝜇 ∈ Name such that 𝑛 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = 𝜇 for all 𝜎 𝑟 ∈ St[Var \ PVar].

  and the fourth clause of used -(𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ). • The second case uses Lemma C.6-(2) (applied to used -(𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ) and dom(𝜉 ′ 𝑛 ) ∩ dom(𝜉 ′′ 𝑛 ) = ∅) and the fourth clause of used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ). • The third case uses Lemma C.6-(2) (applied to used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ) and used -(𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 )). Finally, we put the three results together. Define 𝑓 : St □ [Name] → St as 𝑓 (𝜉 𝑛 ) ≜ 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ (𝜆𝑣 ∈ Var \ (PVar ∪ dom(𝜉 𝑛 )). 1). Then, we obtain the desired equation as follows:

  is continuous. Then, for all 𝜎 𝜃 ∈ St[𝜃 ] and 𝜎 𝑛 ∈ St[Name], ∇ 𝜃 𝑝 ⟨rv (𝜋 ) ⟩ 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) = 0. (20) Proof. Consider 𝑐 and 𝜋 that satisfies the given conditions. Fix 𝜎 𝑛 ∈ St[Name]. Let 𝑓 : St[𝜃 ] → R be the function in Eq. (

  Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023. Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference 12:55

  prs ⟨Name\rv (𝜋 ) ⟩ □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = prs ⟨Name\rv (𝜋 ) ⟩ □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), where 𝜉 𝑛 ≜ vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ). Proof. Fix a reparameterisation plan 𝜋. The proof proceeds by induction on the structure of 𝑐. Let 𝜎 𝑝 ∈ St[PVar], and 𝜉 𝑛 ∈ St □ [Name]. Assume that 𝑐 uses only 𝜆𝑦.𝑦 in the third argument of its sample commands, and used (𝑐 𝜋 , 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) for some 𝜎 𝑟 . Let 𝜎 ≜ 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 and 𝑆 ≜ Name \ rv(𝜋). Then, we simply have used (𝑐 𝜋 , 𝜎, 𝜉 𝑛 ). Cases 𝑐 ≡ skip, 𝑐 ≡ (𝑥 := 𝑒), or 𝑐 ≡ obs(𝑑, 𝑟 ). In this case, 𝑐 𝜎 (cnt 𝜇 ) = 𝜎 (cnt 𝜇 ) for all 𝜎 ∈ St and 𝜇 ∈ Name. So dom(𝜉 𝑛 ) = dom(𝜉 𝑛 ) = ∅ and thus 𝜉 𝑛 = 𝜉 𝑛 . We also know 𝑐 𝜋 ≡ 𝑐. From these, all of the three conclusions follow immediately. Case 𝑐 ≡ (𝑥 := sam(𝑛, 𝑑, 𝜆𝑦.𝑒)). Since fv(𝑛) ⊆ PVar, there exists 𝜇 ∈ Name such that 𝑛 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = 𝜇 for all 𝜎 𝑟 ∈ St[Var \ PVar]. So, for all 𝜎 𝑟 ∈ St[Var \ PVar] and 𝜇 ′ ∈ Name \ {𝜇},

  𝑣) = fv(𝑏) 𝑐 ∩ 𝑝 ′ (𝑣) ∩ 𝑝 ′′ (𝑣) for all 𝑣 ∈ Var. (24) Also, 𝑐 = cond ( 𝑏 , 𝑐 ′ , 𝑐 ′′ ). To prove the conclusion, let 𝑣 ∈ Var. It suffices to apply Lemma F.8 to 𝑓 = 𝑐 ′ , 𝑔 = 𝑐 ′′ , 𝐾 = 𝑝 (𝑣), 𝐿 = {𝑣 }, and 𝑏. What remains is to show the preconditions of the lemma: (a) |= Φ( 𝑐 ′ , 𝑝 (𝑣), {𝑣 }). (b) |= Φ( 𝑐 ′′ , 𝑝 (𝑣), {𝑣 }).

  ∈N is an 𝜔-chain. • For all 𝜏 ∈ St[𝑝 ′ ∞ (𝑣) 𝑐 ] and 𝑛 ∈ N, the set {𝜎 ′ ∈ St[𝑝 ′ ∞ (𝑣)] | 𝑡 ′ 𝑛 (𝜎 ′ ⊕ 𝜏) ∈ St} is ∅ or St[𝑝 ′ ∞ (𝑣)]. • For all 𝑛 ∈ N, we have |= Φ(𝑡 ′ 𝑛 , 𝑝 ′ ∞ (𝑣), {𝑣 }). The first precondition was already observed in §3. The third one holds by the assumption that 𝑡 ′ 𝑛 ∈ 𝑇 ′ for all 𝑛 ∈ N. For the second one, it is enough to show the next two statements: (i) For all 𝑈 ⊆ Var with 𝑈 ⊇ 𝑉 ′ ∞ , and for all 𝜏 ∈ St[𝑈 ] and 𝑛 ∈ N, the next set is ∅ or St[𝑈 𝑐 ]: {𝜎 ′ ∈ St[𝑈 𝑐 ] | 𝑡 ′ 𝑛 (𝜎 ′ ⊕ 𝜏) ∈ St}.

  Consider 𝑈 ⊆ Var, 𝜏 ∈ St[𝑈 ], and 𝑛 ∈ N such that 𝑈 ⊇ 𝑉 ′ ∞ . Let Σ ≜ {𝜎 ′ ∈ St[𝑈 𝑐 ] | 𝑡 ′ 𝑛 (𝜎 ′ ⊕ 𝜏) ∈ St}. If Σ = ∅, there is nothing left to prove. So assume Σ ≠ ∅. To prove Σ = St[𝑈 𝑐 ], we need to show that 𝑡 ′ 𝑛 (𝜎 ′ ⊕ 𝜏) ∈ St for any 𝜎 ′ ∈ St[𝑈 𝑐 ]. Choose 𝜎 ′ ∈ St[𝑈 𝑐 ]. We show 𝑡 ′ 𝑛 (𝜎 ′ ⊕ 𝜏) ∈ St using the next two claims: (iii) For all 𝑛 ∈ N and 𝜎 ∈ St,𝑡 ′ 𝑛 (𝜎) = 𝑐 (𝑖) 0 𝜎 if 𝑖 ∈ 𝐼 𝑛 (𝜎) ⊥ otherwise,(26)where𝑐 (𝑖) 0 ≜ (skip; 𝑐 0 ; • • • ; 𝑐 0 ) that has 𝑖 copies of 𝑐 0 , and 𝐼 𝑛 (𝜎) ≜ {𝑖 ∈ [0, 𝑛 -1] | 𝑐 (𝑖) 0 𝜎 ∈ St ∧ 𝑏 ( 𝑐 (𝑖) 0 𝜎) = false ∧ 𝑏 ( 𝑐 (𝑖-1) 0 𝜎) = • • • = 𝑏 ( 𝑐 (0)0 𝜎) = true}. Note that Eq. (26) is well-defined since 𝐼 𝑛 (𝜎) has at most one element. (iv) For all 𝑛 ∈ N, |= Δ( 𝑐 (𝑛) 0 , 𝑉 ′ ∞ , fv(𝑏)). We give the proof of the two claims below, and for now we just assume them.Since Σ ≠ ∅, there is some𝜎 ′′ ∈ St[𝑈 𝑐 ] such that 𝑡 ′ 𝑛 (𝜎 ′′ ⊕ 𝜏) ∈ St. Since 𝑡 ′ 𝑛 (𝜎 ′′ ⊕ 𝜏) ∈ St, (iii) implies that 𝑡 ′ 𝑛 (𝜎 ′′ ⊕ 𝜏) = 𝑐 (𝑚) 0 (𝜎 ′′ ⊕ 𝜏) ∈ St for some 𝑚 ∈ 𝐼 𝑛 (𝜎 ′′ ⊕ 𝜏). Since 𝜎 ′ ⊕ 𝜏 ∼ 𝑉 ′ ∞ 𝜎 ′′ ⊕ 𝜏 (by 𝑈 ⊇ 𝑉 ′ ∞ ) and 𝑐 (𝑖) 0 (𝜎 ′′ ⊕ 𝜏) ∈ St for all 𝑖 ∈ [0, 𝑚] (by 𝑐 (𝑚) 0 (𝜎 ′′ ⊕ 𝜏) ∈ St), (iv) implies that 𝑐 (𝑖) 0 (𝜎 ′ ⊕ 𝜏) ∈ St and 𝑐 (𝑖) 0 (𝜎 ′ ⊕ 𝜏) ∼ fv (𝑏) 𝑐 (𝑖) 0 (𝜎 ′′ ⊕ 𝜏) for all 𝑖 ∈ [0, 𝑚].By combining these with 𝑚 ∈ 𝐼 𝑛 (𝜎 ′′ ⊕ 𝜏), we get 𝑚 ∈ 𝐼 𝑛 (𝜎 ′ ⊕ 𝜏). Hence, by (iii), we have𝑡 ′ 𝑛 (𝜎 ′ ⊕ 𝜏) = 𝑐 (𝑚) 0 (𝜎 ′ ⊕ 𝜏) ∈ St.

Case

  𝑓 (𝜎 ⊕ 𝜏) ∉ St for all 𝜎 ∈ St[𝐾]. In this case, we set 𝜏 1 ≜ 𝜏 and pick any 𝜏 2 ∈ St[𝐿 𝑐 ]. Then, for all 𝜎 ∈ St[𝐾], ℎ(𝜎) and (ℎ 2 • ℎ 1 ) (𝜎) are both undefined, as desired. Note that the latter term is undefined since ℎ 1 (𝜎) is undefined.Case 𝑓 (𝜎 ′ ⊕ 𝜏) ∈ St for some 𝜎 ′ ∈ St[𝐾]. In this case, we set 𝜏 1 ≜ 𝜏 and 𝜏 2 ≜ 𝜋 Var,𝐿 𝑐 (𝑓 (𝜎 ′ ⊕ 𝜏)).To show ℎ = ℎ 2 • ℎ 1 , consider any 𝜎 ∈ St[𝐾]. If 𝑓 (𝜎 ⊕ 𝜏) ∉ St, then by the same argument for the above case, ℎ(𝜎) and (ℎ 2 • ℎ 1 ) (𝜎) are both undefined. So, assume that 𝑓 (𝜎 ⊕ 𝜏) ∈ St. Then,ℎ(𝜎) = 𝜋 Var,𝑀 (𝑔(𝜎 1 )) if 𝑔(𝜎 1 ) ∈ St undefined otherwise, (ℎ 2 • ℎ 1 ) (𝜎) = 𝜋 Var,𝑀 (𝑔(𝜎 2 )) if 𝑔(𝜎 2 ) ∈ St undefined 𝑓 (𝜎 ⊕ 𝜏) ∈ St, 𝜎 2 = 𝜋 Var,𝐿 (𝑓 (𝜎 ⊕ 𝜏)) ⊕ 𝜋 Var,𝐿 𝑐 (𝑓 (𝜎 ′ ⊕ 𝜏)) ∈ St.Our goal is to show that ℎ(𝜎) and (ℎ 2 • ℎ 1 ) (𝜎) are both undefined, or they are both defined and are the same. By |= Δ(𝑔, 𝐿 ′ , 𝑀) and Eq. (27), it suffices to show 𝜎 1 ∼ 𝐿 ′ 𝜎 2 . To prove this, we show a stronger statement:𝜎 1 ∼ 𝐿 𝜎 2 and 𝜎 1 ∼ 𝐿 ′ \𝐿 𝜎 2 . The former relation holds since 𝜋 Var,𝐿 (𝜎 1 ) = 𝜋 Var,𝐿 (𝑓 (𝜎 ⊕ 𝜏)) = 𝜋 Var,𝐿 (𝜎 2 ). The latter relation is equivalent to 𝑓 (𝜎 ⊕ 𝜏) ∼ 𝐿 ′ \𝐿 𝑓 (𝜎 ′ ⊕ 𝜏), and this holds by |= Δ(𝑓 , 𝐾 𝑐 , 𝐿 ′ \ 𝐿) and 𝜎 ⊕ 𝜏 ∼ 𝐾 𝑐 𝜎 ′ ⊕ 𝜏. Hence, ℎ = ℎ 2 • ℎ 1 as desired. □ Lemma F.8 (Conditional). Let 𝑓 , 𝑓 ′ ∈ [St → St ⊥ ]and 𝐾, 𝐿 ⊆ Var. Then, for any boolean expression 𝑏,|= Φ(𝑓 , 𝐾, 𝐿) ∧ |= Φ(𝑓 ′ , 𝐾, 𝐿) ∧ (𝐾 𝑐 ⊇ fv(𝑏)) =⇒ |= Φ(cond ( 𝑏 , 𝑓 , 𝑓 ′ ), 𝐾, 𝐿).Proof. Let 𝑓 , 𝑓 ′ , 𝐾, 𝐿, and 𝑏 be the functions, sets and a boolean expression such that |= Φ(𝑓 , 𝐾, 𝐿), |= Φ(𝑓 ′ , 𝐾, 𝐿), and 𝐾 𝑐 ⊇ fv(𝑏).Consider 𝜏 ∈ St[𝐾 𝑐 ]. Define 𝑓 ′′ ≜ cond ( 𝑏 , 𝑓 , 𝑓 ′ ), and also partial functions 𝑔, 𝑔 ′ , and 𝑔 ′′ as follows:𝑔 : St[𝐾] ⇀ St[𝐿], 𝑔(𝜎) ≜ (𝜋 Var,𝐿 • 𝑓 ) (𝜎 ⊕ 𝜏) if 𝑓 (𝜎 ⊕ 𝜏) ∈ St undefined otherwise, 𝑔 ′ : St[𝐾] ⇀ St[𝐿], 𝑔 ′ (𝜎) ≜ (𝜋 Var,𝐿 • 𝑓 ′ ) (𝜎 ⊕ 𝜏) if 𝑓 ′ (𝜎 ⊕ 𝜏) ∈ St undefined otherwise, 𝑔 ′′ : St[𝐾] ⇀ St[𝐿], 𝑔 ′′ (𝜎) ≜ (𝜋 Var,𝐿 • 𝑓 ) (𝜎 ⊕ 𝜏) if 𝑓 (𝜎 ⊕ 𝜏) ∈ St undefined otherwise, We should show 𝑔 ′′ ∈ 𝜙 𝐾,𝐿 . Since 𝐾 𝑐 ⊇ fv(𝑏), either 𝑏 (𝜎 ⊕ 𝜏) = true for all 𝜎 ∈ St[𝐾] or 𝑏 (𝜎 ⊕ 𝜏) = false for all 𝜎 ∈ St[𝐾].In the former case, 𝑔 ′′ = 𝑔, and in the latter case, 𝑔 ′′ = 𝑔 ′ . Since both 𝑔 and 𝑔 ′ are in 𝜙 𝐾,𝐿 , we have the desired 𝑔 ′′ ∈ 𝜙 𝐾,𝐿 in both cases. □ Lemma F.9 (Loop; base). Let 𝐾, 𝐿 ⊆ Var. Then,|= Φ((𝜆𝜎 ∈ St. ⊥), 𝐾, 𝐿). Proof. Consider 𝜏 ∈ St[𝐾 𝑐 ]. Define a partial function 𝑔 : St[𝐾] ⇀ St[𝐿] by 𝑔(𝜎) ≜ (𝜋 Var,𝐿 • (𝜆𝜎 ∈ St.⊥)) (𝜎) if (𝜆𝜎 ∈ St.⊥) (𝜎) ∈ St undefined otherwise = undefined.Then, 𝑔 ∈ 𝜙 𝐾,𝐿 by Assumption 7. □ Lemma F.10 (Loop; limit). Let 𝐾, 𝐿 ⊆ Var and {𝑓 𝑛 ∈ [St → St ⊥ ]} 𝑛 ∈N be an 𝜔-chain (i.e., 𝑓 𝑛 ⊑ 𝑓 𝑛+1 for all 𝑛 ∈ N). Here we write 𝑓 ⊑ 𝑔 if 𝑓 (𝜎) ⊑ 𝑔(𝜎) for all 𝜎 ∈ St. Suppose that for any 𝜏 ∈ St[𝐾 𝑐 ] and 𝑛 ∈ N, the set {𝜎 ∈ St[𝐾] | 𝑓 𝑛 (𝜎 ⊕ 𝜏) ∈ St} is either ∅ or St[𝐾]. Then, 𝑛 ∈N |= Φ(𝑓 𝑛 , 𝐾, 𝐿) =⇒ |= Φ( 𝑛 ∈N 𝑓 𝑛 , 𝐾, 𝐿). Proof. Consider an 𝜔-chain {𝑓 𝑛 ∈ [St → St ⊥ ]} 𝑛 ∈N such that |= Φ(𝑓 𝑛 , 𝐾, 𝐿) for all 𝑛. Pick an arbitrary 𝜏 ∈ St[𝐾 𝑐 ]. Let 𝑓 ∞ ≜ 𝑛 ∈N 𝑓 𝑛 , and define partial functions 𝑔 ∞ and 𝑔 𝑛 for all 𝑛 as follows: 𝑔 ∞ : St[𝐾] ⇀ St[𝐿], 𝑔 ∞ (𝜎) ≜ (𝜋 Var,𝐿 • 𝑓 ∞ ) (𝜎 ⊕ 𝜏) if 𝑓 ∞ (𝜎 ⊕ 𝜏) ∈ St undefined otherwise, 𝑔 𝑛 : St[𝐾] ⇀ St[𝐿], 𝑔 𝑛 (𝜎) ≜ (𝜋 Var,𝐿 • 𝑓 𝑛 ) (𝜎 ⊕ 𝜏) if 𝑓 𝑛 (𝜎 ⊕ 𝜏) ∈ St undefined otherwise.

  𝑛 ≜ {𝜎 ∈ St[𝐾] | 𝑓 𝑛 (𝜎 ⊕ 𝜏) ∈ St}. Note that this implies the desired 𝑔 ∞ ∈ 𝜙 𝐾,𝐿 because 𝑔 𝑛 ∈ 𝜙 𝐾,𝐿 for all 𝑛 ∈ N. If Σ 𝑛 = ∅ for all 𝑛 ∈ N, then 𝑔 = 𝑔 𝑛 for any 𝑛, since both 𝑔 and 𝑔 𝑛 are the same empty partial function. Otherwise, by the assumption of the lemma, Σ 𝑛 = St[𝐾] for some 𝑛. This means that 𝑔 𝑛 is the total function, and so 𝑔 𝑛 = 𝑔 𝑚 for all 𝑚 ≥ 𝑛, which implies that 𝑔 = 𝑔 𝑛 , as desired. □ G DEFERRED RESULTS IN §5.3 G.1 Proof of Theorem 5.10

  Case of Assumption 4. Let 𝐾, 𝐿 0 , 𝐿 1 ⊆ Var such that 𝐿 0 ∩ 𝐿 1 = ∅. Consider 𝑓 0 , 𝑔 0 ∈ [St[𝐾] ⇀ St[𝐿 0 ]] and 𝑓 1 , 𝑔 1 ∈ [St[𝐾] ⇀ St[𝐿 1 ]] such that all of the following hold:𝑓 0 ∈ 𝜙 (𝑑) 𝐾,𝐿 0 , 𝑓 1 ∈ 𝜙 (𝑑) 𝐾,𝐿 1 , 𝑔 0 ∈ 𝜙 (𝑙) 𝐾,𝐿 0 ,and𝑔 1 ∈ 𝜙 (𝑙) 𝐾,𝐿 1 . Let 𝐿 ≜ 𝐿 0 ∪ 𝐿 1 ; 𝑓 : St[𝐾] ⇀ St[𝐿], 𝑓 (𝜎) ≜ 𝑓 0 (𝜎) ⊕ 𝑓 1 (𝜎) if 𝜎 ∈ dom(𝑓 0 ) ∩ dom(𝑓 1 ), undefined otherwise; 𝑔 : St[𝐾] ⇀ St[𝐿], 𝑔(𝜎) ≜ 𝑔 0 (𝜎) ⊕ 𝑔 1 (𝜎) if 𝜎 ∈ dom(𝑔 0 ) ∩ dom(𝑔 1), undefined otherwise.

  [St[𝐾] ⇀ St[𝐿 0 ]] and [St[𝐾] ⇀ St[𝐿 1 ]], respectively, such that their domains are open and contain 0, and for all 𝑖 ∈ {0, 1}, lim𝜎 ′ →0 ∥𝑓 𝑖 (𝜎 + 𝜎 ′ ) -𝑓 𝑖 (𝜎)ℎ 𝑖 (𝜎 ′ ) ∥ 2 ∥𝜎 ′ ∥ 2 = 0. Let ℎ be the linear function in [St[𝐾] ⇀ St[𝐿]] defined by ℎ(𝜎) ≜ ℎ 0 (𝜎) ⊕ ℎ 1 (𝜎) if 𝜎 ∈ dom(ℎ 0 ) ∩ dom(ℎ 1 ); undefined otherwise.Then, dom(ℎ) is open and contains 𝜎. Furthermore, lim𝜎 ′ →0 ∥𝑓 (𝜎 + 𝜎 ′ ) -𝑓 (𝜎)ℎ(𝜎 ′ ) ∥ 2 ∥𝜎 ′ ∥ 2 = lim 𝜎 ′ →0 ∥𝑓 0 (𝜎 + 𝜎 ′ ) -𝑓 0 (𝜎)ℎ 0 (𝜎 ′ ) ∥ 2 2 + ∥ 𝑓 1 (𝜎 + 𝜎 ′ ) -𝑓 1 (𝜎)ℎ 1 (𝜎 ′ ) ∥ 2 2 ∥𝜎 ′ ∥ 2 = lim 𝜎 ′ →0 ∥𝑓 0 (𝜎 + 𝜎 ′ ) -𝑓 0 (𝜎)ℎ 0 (𝜎 ′ ) ∥ 2 ∥𝜎 ′ ∥ 2 2 + ∥𝑓 1 (𝜎 + 𝜎 ′ ) -𝑓 1 (𝜎)ℎ 1 (𝜎 ′ ) ∥ 2 ∥𝜎 ′ 𝜎 + 𝜎 ′ ) -𝑓 0 (𝜎)ℎ 0 (𝜎 ′ ) ∥ 2 ∥𝜎 ′ ∥ 2 2

  Case of Assumption 5. Consider 𝐾, 𝐾 ′ , 𝐿 ⊆ Var with 𝐾 ⊆ 𝐾 ′ , and 𝜏 ∈ St[𝐾 ′ \ 𝐾]. Let 𝑓 and 𝑔 be partial functions in [St[𝐾 ′ ] ⇀ St[𝐿]] such that 𝑓 ∈ 𝜙(𝑑) 𝐾 ′ ,𝐿 and 𝑔 ∈ 𝜙 (𝑙) 𝐾 ′ ,𝐿 . Let𝑓 1 : St[𝐾] ⇀ St[𝐿], 𝑓 1 (𝜎) ≜ 𝑓 (𝜎 ⊕ 𝜏) if 𝜎 ⊕ 𝜏 ∈ dom(𝑓 ) undefined otherwise, 𝑔 1 : St[𝐾] ⇀ St[𝐿], 𝑔 1 (𝜎) ≜ 𝑔(𝜎 ⊕ 𝜏) if 𝜎 ⊕ 𝜏 ∈ dom(𝑔) undefined otherwise.We should show that 𝑓 1 and 𝑔 1 satisfy 𝜙(𝑑) 𝐾,𝐿 and 𝜙 (𝑙) 𝐾,𝐿 , respectively. Note that dom(𝑓 1 ) = {𝜎 | 𝜎 ⊕ 𝜏 ∈ dom(𝑓 )} and dom(𝑔 1 ) = {𝜎 | 𝜎 ⊕ 𝜏 ∈ dom(𝑔)}. Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023. Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference 12:81 These two sets are open since dom(𝑓 ) and dom(𝑔) are open and for any open 𝑂, the slice {𝜎∈ St[𝐾] | 𝜎 ⊕ 𝜏 ∈ 𝑂 } is open. Let 𝜎 0 ∈ dom(𝑓 1 )and 𝜎 1 ∈ dom(𝑔 1 ). We will show that 𝑓 1 is differentiable at 𝜎 0 , and 𝑔 1 is Lipschitz in an open neighbourhood of 𝜎 1 .Since 𝑓 is differentiable and 𝜎 0 ⊕ 𝜏 ∈ dom(𝑓 ), there exists a linear map ℎ :St[𝐾 ′ ] ⇀ St[𝐿] such that dom(ℎ)is open and contains 0, and lim𝜎 ′ →0 ∥𝑓 (𝜎 0 ⊕ 𝜏 + 𝜎 ′ ) -𝑓 (𝜎 0 ⊕ 𝜏)ℎ(𝜎 ′ ) ∥ 2 ∥𝜎 ′ ∥ 2 = 0. Let 𝜏 0 ≜ 𝜆𝑣 ∈ 𝐾 ′ \ 𝐾 . 0,and ℎ 1 be the partial function from St[𝐾] to St[𝐿] defined by ℎ 1 (𝜎) ≜ ℎ(𝜎 ⊕ 𝜏 0 ) if 𝜎 ⊕ 𝜏 0 ∈ dom(ℎ), undefined otherwise. Then, ℎ 1 is linear, its domain is open (since taking a slice of an open set in St[𝐾] R |𝐾 ′ | by fixing some coordinate variables gives an open set), and lim𝜎 ′′ →0 ∥𝑓 1 (𝜎 0 + 𝜎 ′′ ) -𝑓 1 (𝜎 0 )ℎ 1 (𝜎 ′′ ) ∥ 2 ∥𝜎 ′′ ∥ 2 = lim 𝜎 ′′ →0 ∥𝑓 (𝜎 0 ⊕ 𝜏 + 𝜎 ′′ ⊕ 𝜏 0 ) -𝑓 (𝜎 0 ⊕ 𝜏)ℎ(𝜎 ′′ ⊕ 𝜏 0 ) ∥ 2 ∥𝜎 ′′ ⊕ 𝜏 0 ∥ 2 = 0.This means that 𝑓 1 is differentiable at 𝜎 0 .Since 𝑔 is locally Lipschitz and 𝜎 1 ⊕ 𝜏 ∈ dom(𝑔), there exists an open subset 𝑂 of dom(𝑔) such that 𝑂 contains 𝜎 1 ⊕ 𝜏 and 𝑔 is Lipschitz in 𝑂, that is, there exists a real number 𝐵 > 0 such that∥𝑔(𝜎) -𝑔(𝜎 ′ ) ∥ 2 ≤ 𝐵 • ∥𝜎 -𝜎 ′ ∥ 2 for all 𝜎, 𝜎 ′ ∈ 𝑂. Let 𝑂 ′ ≜ {𝜎 ∈ St[𝐾] | 𝜎 ⊕ 𝜏 ∈ 𝑂 }.Then, 𝑂 ′ is open, and it contains 𝜎 1 . Furthermore, for all 𝜎, 𝜎 ′ ∈ 𝑂 ′ , ∥𝑔 1 (𝜎) -𝑔 1 (𝜎 ′ ) ∥ 2 = ∥𝑔(𝜎 ⊕ 𝜏) -𝑔(𝜎 ′ ⊕ 𝜏) ∥ 2 ≤ 𝐵 • ∥𝜎 ⊕ 𝜏 -𝜎 ′ ⊕ 𝜏 ∥ 2 = 𝐵 • ∥𝜎 -𝜎 ′ ∥ 2 .

  𝐾, 𝐿, 𝑀 ⊆ Var, 𝑓 ∈ [St[𝐾] ⇀ St[𝐿]], and 𝑔 ∈ [St[𝐿] ⇀ St[𝑀]].

  Definition 3.1. An always-terminating command 𝑐 does not have a double-sampling error if for any 𝜎 ∈ St, we have 𝑐 𝜎 (cnt 𝜇 ) -𝜎 (cnt 𝜇 ) ≤ 1 for all 𝜇 ∈ Name.

□

Example 3.2 (Density semantics). Consider the following state 𝜎 ∈

  𝑛 ) as an estimate of ∇ 𝜃 L (𝜎 𝜃 ). The presented results hold for this more general case as well. sample command in 𝑐 𝑔 has 𝜆𝑦.𝑦 as its third argument, and 𝑐 𝑔 does not have observe commands. Then, for all 𝜎 𝜃 ∈ St[𝜃 ], ∇ 𝜃 L (𝜎 𝜃 ) = E 𝑝 𝑐𝑔 𝜋 ,𝜎 𝜃 ( σ𝑛 ) [grad_est(𝜎 𝜃 ; σ𝑛 )] For all 𝜎 𝑛 ∈ St[Name], the below functions on St[𝜃 ] are differentiable in 𝜃 jointly: 𝜎 𝜃 ↦ -→ 𝑣 𝑐 𝑔 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ), 𝜎 𝜃 ↦ -→ 𝑝 ⟨rv (𝜋 ) ⟩ For all 𝜎 𝜃 ∈ St[𝜃 ] and 𝜎 𝑛 ∈ St[Name], we have ∇ 𝜃 𝑝 ⟨rv (𝜋 ) ⟩

						(9)
	if 𝜋 satisfies the following requirements:
	(R1) 𝜋 is valid and simple.
	(R2) The below functions from St[𝜃 ] × St[Name] to (0, ∞) are differentiable in 𝜃 ∪ rv(𝜋) jointly:
	(𝜎 𝜃 , 𝜎 𝑛 ) ↦ -→ 𝑝 𝑐 𝑚 ,𝜎 𝜃 (𝜎 𝑛 ),	(𝜎 𝜃 , 𝜎 𝑛 ) ↦ -→ 𝑝 ⟨rv (𝜋 ) ⟩ 𝑐 𝑔 ,𝜎 𝜃 (𝜎 𝑛 ),	(𝜎 𝜃 , 𝜎 𝑛 ) ↦ -→ 𝑝 ⟨Name\rv (𝜋 ) ⟩ 𝑐 𝑔 ,𝜎 𝜃	(𝜎 𝑛 ).
	(R3) 𝑐 𝑔	𝜋 ,𝜎 𝜃	(𝜎 𝑛 ),		𝜎 𝜃 ↦ -→ 𝑝 ⟨Name\rv (𝜋 ) ⟩ 𝑐 𝑔 𝜋 ,𝜎 𝜃	(𝜎 𝑛 ).
	(R4) 𝑐 𝑔 (R5) The below equations hold for all 𝜎 𝜃 ∈ St[𝜃 ]:	𝜋 ,𝜎 𝜃	(𝜎 𝑛 ) = 0.
	10 In practice, the SPGE often draws a fixed number of independent samples σ (1) 𝑛 , . . . , σ (𝑀 ) 𝑛 1 𝑀 𝑀 𝑖=1 grad_est(𝜎 𝜃 ; σ (𝑖 )	from 𝑝 𝑐𝑔 𝜋 ,𝜎 𝜃 and computes

  𝑚 for 𝑉 ⊆ R 𝑛 is Lipschitz if there is 𝐶 > 0 such that ∥𝑓 (𝑥) -𝑓 (𝑥 ′ ) ∥ 2 ≤ 𝐶 ∥𝑥 -𝑥 ′ ∥ 2 for all 𝑥, 𝑥 ′ ∈ 𝑉 ; and 𝑓 is locally Lipschitz if for all 𝑥 ∈ R 𝑛 , there is an open neighborhood 𝑈 ⊆ R 𝑛 of 𝑥 such that 𝑓 | 𝑈 : 𝑈 → R 𝑚 is Lipschitz. Further, 𝑓 : St[𝐾] → R 𝑚 for 𝐾 ⊆ Var is locally Lipschitz in 𝐾 ′ ⊆ 𝐾 jointly if for any 𝜏 ∈ St[𝐾 \ 𝐾 ′ ], 𝑓 | [𝜏 ] : St[𝐾 ′ ] → R 𝑚 is locally Lipschitz, where

	𝑓 | [𝜏 ] (𝜎) ≜ 𝑓 (𝜎 ⊕ 𝜏). Although differentiability does not imply local Lipschitzness, continuous differ-
	entiability does. Since most differentiable functions used in practice are continuously differentiable,
	asking for (R2

]. Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

Here 𝑓 : 𝑉 → R

  In the instantiation of our program analysis for differentiability, we use the family 𝜙(𝑑) where for all 𝐾, 𝐿 ⊆ Var, a partial function 𝑓 : St[𝐾] → St[𝐿] belongs to 𝜙(𝑑)

	1.1 Family of Smoothness Predicates. Our program analysis assumes that a target smoothness
	property is specified in terms of a family of predicates,
	𝜙 = (𝜙 𝐾,𝐿 : 𝐾, 𝐿 ⊆ Var),
	where 𝜙 𝐾,𝐿 is a set of partial functions from St[𝐾] to St[𝐿] (i.e., 𝜙 𝐾,𝐿 ⊆ [St[𝐾] ⇀ St[𝐿]]).
	Example 5.1 (Differentiability). 𝐾,𝐿
	if and only if (i) dom(𝑓 ) is open and (ii) 𝑓 is (jointly) differentiable in its domain.	□
	At first, one may wonder why we use a family of 𝜙 𝐾,𝐿 predicates instead of a single predicate 𝜙 0

  Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

	Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference	12:17

  .1.2 Smoothness Abstraction. Based on the family 𝜙, we build a predicate Φ that captures the smoothness of commands. The Φ constrains functions from St to St ⊥ , unlike 𝜙 𝐾,𝐿 . For 𝐾, 𝐿 ⊆ Var with 𝐾 ⊇ 𝐿, define 𝜋 𝐾,𝐿 to be the projection from St[𝐾] to St[𝐿]. Definition 5.2. The smoothness abstraction Φ is the predicate over a function 𝑓 ∈ [St → St ⊥ ] and variable sets 𝐾, 𝐿 ⊆ Var. It is satisfied by (𝑓 , 𝐾, 𝐿) if for all 𝜏 ∈ St[Var \ 𝐾], the predicate 𝜙 𝐾,𝐿 holds for the following partial function 𝑔: St[𝐾] ⇀ St[𝐿]: dom(𝑔) ≜ {𝜎 ∈ St[𝐾] | 𝑓 (𝜎 ⊕ 𝜏) ∈ St} and 𝑔(𝜎) ≜ (𝜋 Var,𝐿 • 𝑓 ) (𝜎 ⊕ 𝜏) for 𝜎 ∈ dom(𝑔).We denote the satisfaction of Φ by |= Φ(𝑓 , 𝐾, 𝐿). □ Note that the function 𝑔 is constructed from 𝑓 by fixing the Var \ 𝐾 part of the input state to 𝜏, and looking at only the 𝐿 part of the output. This construction is precisely the one used in the informal definition of conditional differentiability described above, and its use reflects the fact that our program analysis attempts to prove a conditional smoothness property. 5.1.3 Dependency Abstraction. Our abstract domain has a component for tracking dependency between input-output variables. Dependency here means that a given input variable is used for computing a given output variable. We define a predicate Δ that has a similar format as Φ. Intuitively, Δ(𝑓 , 𝐾, 𝐿) holds if and only if the 𝐿 part of the output of 𝑓 depends at most on the 𝐾 part of the input to 𝑓 . To define Δ formally, for 𝐾 ⊆ Var, let ∼ 𝐾 be the following equivalence relation over states: 𝜎 ∼ 𝐾 𝜎 ′ if and only if 𝜎 (𝑣) = 𝜎 ′ (𝑣) for all 𝑣 ∈ 𝐾. Definition 5.3. The dependency abstraction Δ is the predicate on 𝑓 ∈ [St → St ⊥ ] and 𝐾, 𝐿 ⊆ Var that holds if for all 𝜎, 𝜎 ′ ∈ St with 𝜎 ∼ 𝐾 𝜎 ′ , we have (𝑓 (𝜎) ∈ St ⇐⇒ 𝑓 (𝜎 ′ ) ∈ St) and (𝑓 (𝜎) ∈ St =⇒ 𝑓 (𝜎) ∼ 𝐿 𝑓 (𝜎 ′ )). We denote the satisfaction of Δ by

	|= Δ(𝑓 , 𝐾, 𝐿).	□
	5.1.4 Combined Abstraction. We bring together the two abstractions that we just defined, and
	construct the final abstract domain D ♯ used by our program analysis.
	Intuitively, each element of D ♯ is a predicate on a function 𝑓 ∈ [St → St ⊥ ] expressed by the
	conjunction of the following form: 𝑚 𝑖=1 Φ(𝑓 , 𝐾 𝑖 , 𝐿 𝑖 ) ∧ 𝑛 𝑗=1 Δ(𝑓 , 𝐾 ′ 𝑗 , 𝐿 ′ 𝑗 ).

  Definition 5.4. The abstract domain D ♯ consists of triples (𝑝, 𝑑, 𝑉 ) ∈ [Var → P (Var)] 2 × P (Var), called abstract state, such that 𝑝 (𝑣) ⊇ 𝑑 (𝑣) 𝑐 and 𝑑 (𝑣) ⊇ 𝑉 for all 𝑣 ∈ Var, where -𝑐 is the standard operation for set complement. That is,D ♯ ≜ {(𝑝, 𝑑, 𝑉 ) ∈ [Var → P (Var)] 2 × P (Var) | 𝑝 (𝑣) ⊇ 𝑑 (𝑣) 𝑐 and 𝑑 (𝑣) ⊇ 𝑉 for all 𝑣 ∈ Var}.We order abstract states as follows: (𝑝, 𝑑, 𝑉 ) ⊑ (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) if and only if 𝑉 ⊆ 𝑉 ′ and for all 𝑣 ∈ Var, 𝑝 (𝑣) ⊇ 𝑝 ′ (𝑣) and 𝑑 (𝑣) ⊆ 𝑑 ′ (𝑣). These abstract states are concretised by 𝛾 :D ♯ → P ( [St → St ⊥ ]):Note that the definition of D ♯ contains two conditions. The first condition 𝑝 (𝑣) ⊇ 𝑑 (𝑣) 𝑐 comes from our assumption that if a function does not depend on a variable 𝑢, it is smooth in 𝑢. This and other assumptions of the analysis will be explained shortly in §5.2.2. The other condition 𝑑 (𝑣) ⊇ 𝑉 originates from the relationship that if Δ(𝑓 , 𝐾, {𝑣 }) holds, so does Δ(𝑓 , 𝐾, ∅).

	𝑓 ∈ 𝛾 (𝑝, 𝑑, 𝑉 ) ⇐⇒ |= Δ(𝑓 , 𝑉 , ∅), |= Φ(𝑓 , 𝑝 (𝑣), {𝑣 }), and |= Δ(𝑓 , 𝑑 (𝑣), {𝑣 }) for all 𝑣 ∈ Var. □ (10)

  3) Using 𝑆 and 𝜋 0 , construct a reparameterisation plan 𝜋 ⊑ 𝜋 0 by 𝜋 ≜ 𝜋 0[𝑆], where 𝜋 0 [𝑆] (𝑛, 𝑑, 𝑙) is 𝜋 0 (𝑛, 𝑑, 𝑙) if (𝑛, 𝑑, 𝑙) ∈ dom(𝜋 0 ), 𝑛 = name(𝛼, _), and (𝛼, _) ∈ 𝑆; otherwise, it is undefined. (4) By running the differentiability analysis on 𝑐 𝑔 𝜋 , compute (p 𝑔 , d 𝑔 , V 𝑔 ) ≜ 𝑐 𝑔

			𝜋 ♯	and check
	𝜃 ⊆	p 𝑔 (pr 𝜇 ) ∩	p
	𝜇 ∈Name	𝜇 ∈Name

𝑔 (val 𝜇 ).

  Table 3 lists half of these 14 Pyro examples with their code size and conceptual complexity (see §I for the rest). Experiments were performed on a Macbook Pro with 2.3GHz Core i9 and 32GB RAM.

Table 3 .

 3 Subset of Pyro examples used in experiments and their key features (see §I for the rest). The last five columns show the total number of code lines (excluding comments), loops, sample commands, observe commands, and learnable parameters (declared explicitly by pyro.param or implicitly by a neural network module). Each number is the sum of the counts in the model and guide.

	Name	Probabilistic model	LoC while sam obs param
	spnor	Splitting normal example in Fig. 1	16	0	2	1	2
	sgdef	Deep exponential family	105	0	12	1	12
	dmm	Deep Markov model	112	3	2	1	13
	mhmm	Hidden Markov models	137	1	5	5	12
	scanvi Single-cell annotation using variational inference 147	0	7	2	21
	air	Attend-infer-repeat	174	2	6	1	16
	cvae	Conditional variational autoencoder	205	0	2	1	15

Table 4 .

 4 Results of smoothness analyses. "Manual" and "Ours" denote the number of continuous random variables and learnable parameters in which the density of the program is smooth, computed by hand and by our analyser. "Time" denotes the runtime of our analyser in seconds. "#CRP" denotes the total number of continuous random variables and learnable parameters in the program. -m and -g denote model and guide. We consider {(𝛼, 𝑖) ∈ Name} as one random variable for each 𝛼 ∈ Name. See §I for the rest of results.

		Differentiable	Locally Lipschitz
	Name	Manual Ours Time Manual Ours Time #CRP
	spnor-m	1	1 0.006	1	1 0.009	2
	spnor-g	4	4 0.007	4	4 0.008	4
	sgdef-m	6	6 0.003	6	6 0.006	6
	sgdef-g	18	18 0.016	18	18 0.015	18
	dmm-m	4	4 0.014	10	10 0.016	10
	dmm-g	4	4 0.026	5	5 0.020	5
	mhmm-m	10	10 0.063	10	10 0.075	10
	mhmm-g	6	6 0.007	6	6 0.008	6
	scanvi-m	6	6 0.032	12	12 0.032	12
	scanvi-g	8	8 0.052	15	15 0.058	15
	air-m	1	1 0.108	4	4 0.105	4
	air-g	3	3 0.075	15	15 0.072	16
	cvae-m	3	3 0.025	8	8 0.027	8
	cvae-g	5	5 0.031	9	9 0.023	9

  Table 5 displays the results (only for 7 examples; see §I for the rest) and compares them with 𝜋 0 .

				\ Ours
	Name	Time Sound Sound Unsound #CR #DR
	spnor	0.021	1	0	1	2	0
	sgdef	0.034	6	0	0	6	0
	dmm	0.054	1	0	0	1	0
	mhmm	0.083	2	0	0	2	1
	scanvi 0.143	3	0	0	3	1
	air	0.247	1	0	1	2	1
	cvae	0.063	1	0	0	1	0

variables too); see, for instance, scanvi and cvae. Due to this, the requirement (R2) is not satisfied for these examples even with the empty reparameterisation plan (corresponding to the score estimator); that is, if we use the differentiability requirements (R2) and (R3) to validate the unbiasedness of gradient estimators, even the score estimator cannot be validated for these examples. From manual inspection, we checked that the non-differentiabilities from these examples all arise by the use of locally Lipschitz but non-differentiable operators (e.g., relu and grid_sample). Since many practical models (and guides) use locally Lipschitz but non-differentiable operators, this observation strongly suggests that a right smoothness requirement for validating gradient estimators is not differentiability (which has been used as a standard requirement), but rather local Lipschitzness (e.g., (R2') and (R3')). Variable selector. To evaluate our variable selector, we consider the SPGE variable-selection problem with local Lipschitzness requirements, i.e., the problem that uses (R2') and (R3') in §4.3 instead of (R2) and (R3) in §4.2. We do not consider the original problem (with differentiability requirements), since for many examples the differentiability requirements are not satisfied even by the empty reparameterisation plan (i.e., score estimator) as observed above. For an initial reparameterisation plan 𝜋 0 for the problem, we use the plan given by Pyro's default variable selector: it is defined for all continuous random variables and applies standard reparameterisations (e.g., Eq. (

11

) for a normal distribution). In this settings, we apply our variable selector to the problem on the 14 Pyro examples.

Table 6 .

 6 Gradient estimators for variational inference, and requirements for each estimator. "Req." denotes "Requirement" and "diff. " denotes "differentiable". Recall that 𝑓 𝜃 (𝑧) ≜ log(𝑝 𝑐 𝑚 (𝑧)/𝑝 𝑐 𝑔 ,𝜃 (𝑧)). 𝑓 𝜃 (𝑣 𝜃 (𝑧)) • ∇ 𝜃 log 𝑞 𝜃 (𝑧) ∇ 𝜃 𝑓 𝜃 (𝑣 𝜃 (𝑧)) ∇ 𝜃 𝑓 𝜃 (𝑣 𝜃 (𝑧)) + 𝑓 𝜃 (𝑣 𝜃 (𝑧)) • ∇ 𝜃 log 𝑞 𝜃 (𝑧)

			SCE	PGE	SPGE
	Setup 𝑞 𝜃 (𝑧)	𝑝 𝑐 𝑔 ,𝜃 (𝑧)	𝑝 𝑐 ′ 𝑔 (𝑧)	𝑝 𝑐 ′′ 𝑔 ,𝜃 (𝑧)
		𝑣 𝜃 (𝑧)	𝑧	𝑣 𝑐 ′ 𝑔 ,𝜃 (𝑧)	𝑣 𝑐 ′′ 𝑔 ,𝜃 (𝑧)
		𝑔 𝜃 (𝑧)
	Req.	𝑞 𝜃 (𝑧)	diff. in 𝜃	-	diff. in 𝜃
		𝑣 𝜃 (𝑧)	-	diff. in 𝜃	diff. in 𝜃
		𝑝 𝑐

𝑚 (𝑧) -diff. in 𝜃 and 𝑧 diff. in 𝜃 and "selected" 𝑧 𝑖 's 𝑝 𝑐 𝑔 ,𝜃 (𝑧) -diff. in 𝜃 and 𝑧 diff. in 𝜃 and "selected" 𝑧 𝑖 's

Table 6

 6 We introduce several definitions, state lemmas, and prove Theorem 4.2 using the lemmas. We prove the lemmas in §C.2 and §C.3. Recall the partition Var = PVar ⊎ Name ⊎ AVar of Var. We use the following letters to denote the values of each part: 𝜎 𝑝 ∈ St[PVar], 𝜎 𝑛 ∈ St[Name], and 𝜎 𝑎 ∈ St[AVar]. Based on the partition, we define the next functions: prs

	Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference	12:35
	C DEFERRED RESULTS IN §4.1
	C.1 Proof of Theorem 4.2

compares key aspects of the three gradient estimators (SCE, PGE, and SPGE) explained in §2. Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

  𝜎, 𝜉 𝑛 ) is a predicate for a command 𝑐, 𝜎 ∈ St, and 𝜉 𝑛 ∈ St □ [Name], defined by used (𝑐, 𝜎, 𝜉 𝑛 ) ⇐⇒ noerr (𝑐, 𝜎) ∧ 𝜎

used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) 𝜆𝑥 ∈ PVar. 0 otherwise,

where used (𝑐,

  𝑐,𝜎 𝜃 , 𝑣 𝑐,𝜎 𝜃 , prs, and vals. The second equality holds by Lemma C.2 (applied to 𝑐). The third equality follows from Lemma C.3. The fourth equality holds by Lemma C.2 (applied to 𝑐 𝜋 ). The fifth equality holds by Eq. (15) and the definitions of 𝑝 𝑐 𝜋 ,𝜎 𝜃 , 𝑣 𝑐 𝜋 ,𝜎 𝜃 , prs, and vals. Note that the same equational reasoning with the reverse direction can be used to prove the claimed equation of the theorem when the integral on the RHS of the equation is defined. Consider 𝜎 𝑟 ∈ St[Var \ (dom(𝜎 𝑝 ) ∪ dom(𝜉 𝑛 ))] such that used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ). We want to show that prs □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), and pvars □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) do not depend on the choice of 𝜎 𝑟 . To do so, consider 𝜎 ′ 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 ′ 𝑟 . Then, it suffices to show that 𝑐 𝜎 (like) • 𝜇 ∈dom(𝜉 𝑛 ) 𝑐 𝜎 (pr 𝜇 ) = 𝑐 𝜎 ′ (like) • 𝜇 ∈dom(𝜉 𝑛 ) 𝑐 𝜎 ′ (pr 𝜇 ), 𝑐 𝜎 (val 𝜇 ) = 𝑐 𝜎 ′ (val 𝜇 ) for all 𝜇 ∈ dom(𝜉 𝑛 ), 𝑐 𝜎 (𝑥) = 𝑐 𝜎 ′ (𝑥) for all 𝑥 ∈ PVar.

	□
	C.2 Proofs of Lemmas C.1 and C.2

Proof of Lemma C.1. Let 𝑐 be a command, 𝜎 𝑝 ∈ St[PVar], and

𝜉 𝑛 ∈ St □ [Name]. 𝑟 ∈ St[Var \ (dom(𝜎 𝑝 ) ∪ dom(𝜉 𝑛 ))] such that used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 ′ 𝑟 , 𝜉 𝑛 ). Let 𝜎 ≜ 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟

and 𝜎 ′ ≜

  The fourth equality holds since 𝜉 ′ 𝑛 ∈ [𝐿 → R] with 𝐿 ≠ Name \ 𝐾 implies 1 [dom(𝜉 𝑛 )⊎dom(𝜉 ′ 𝑛 )=Name] = 0. The fifth equality holds by the definition of St □ [Name] and its underlying measure. Using this result, we obtain the desired equation: ∫ 𝑑𝜎 𝑛 prs(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) • ℎ vals(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓

The first equality holds because noerr (𝑐, 𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝑓 * (𝜎 𝑛 )) implies that there exists a unique 𝐾 ⊆ Name with used (𝑐, 𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝑓 * (𝜎 𝑛 ), 𝜎 𝑛 | 𝐾 ); here we use 𝑓 * (𝜎 𝑛 ) (like) = 1. The second equality holds since Name is finite. The third equality holds because

St[Name] is isomorphic to [𝐾 → R] × [Name \ 𝐾 → R]. * (𝜎 𝑛 )) = ∫ 𝑑𝜎 𝑛 1 [

noerr (𝑐,𝜎 𝑝 ⊕𝜎 𝑛 ⊕𝑓 * (𝜎 𝑛 )) ] • prs(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛 )) • ℎ vals(𝑐) (𝜎 𝑝 , 𝜎 𝑛 , 𝑓 * (𝜎 𝑛

  • If the premise holds, then there exists 𝜎 𝑟 such that used (𝑐,𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ). Since 𝜎 𝑟 (like) = 1 = 𝑓 * (-) (like), 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 and 𝜎 𝑝 ⊕ (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 ) ⊕ 𝑓 * (𝜉 𝑛 ⊕ 𝜉 ′ 𝑛 )coincide on PVar ∪ dom(𝜉 𝑛 ) ∪ {like}. Thus, Lemma C.7 gives the conclusion.Again back to the main equations, we note that the sixth equality holds since dom(𝜉 𝑛 ) = dom(vals □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 )), and the seventh equality follows from the definition of 𝑔. This completes the proof.

□ Lemma C.4. For all commands 𝑐 and states 𝜎 ∈ St such that 𝑐 𝜎 ∈ St, we have 𝑐 𝜎 (cnt 𝜇 ) ≥ 𝜎 (cnt 𝜇 ) and 𝑐 𝜎 (𝜇) = 𝜎 (𝜇)

  for all 𝑎 ∈ {pr 𝜇 , val 𝜇 | 𝜇 ∈ dom(𝜉 ′ 𝑛 )}, we have 𝑎 ∈ {pr 𝜇 , val 𝜇 | 𝜇 ∈ Name \ dom(𝜉 ′′ 𝑛 )}, and we can calculate:

  . . in T . Let 𝑓 ∞ ≜ 𝑛 ∈N 𝑓 𝑛 . Consider 𝜎 ′′ Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference 12:45 𝑓 ∞ (𝜎 0 ) = 𝑓 𝑚 (𝜎 0 ). Then, the assumption used -(𝑓 ∞ , 𝜎 ′′ 0 , 𝜉 ′′ 𝑛 ) implies used -(𝑓 𝑚 , 𝜎 ′′ 0 , 𝜉 ′′ 𝑛 ). This in turn gives 𝜑 (𝑓 𝑚 , 𝜉 ′′ 𝑛 , 𝜎 ′′ 0 , 𝜎 ′′ 1 ) because 𝑓 𝑚 ∈ T . By what we have proved and the definition of 𝑓 ∞ , we have 𝑓 𝑚 (𝜎 ′′ 0 ) = 𝑓 ∞ (𝜎 ′′ 0 ) ∈ St and 𝑓 𝑚 (𝜎 ′′ 1 ) = 𝑓 ∞ (𝜎 ′′ 1 ) ∈ St. Thus, 𝜑 (𝑓 𝑚 , 𝜉 ′′ 𝑛 , 𝜎 ′′ 0 , 𝜎 ′′ 1 ) entails 𝜑 (𝑓 ∞ , 𝜉 ′′ 𝑛 , 𝜎 ′′ 0 , 𝜎 ′′ 1

0 , 𝜎 ′′ 1 ∈ St and 𝜉 ′′ 𝑛 ∈ St □ such that 𝜎 ′′ 0 ∼ 𝜉 ′′ 𝑛 𝜎 ′′ 1 and used -(𝑓 ∞ , 𝜎 ′′ 0 , 𝜉 ′′ 𝑛 ). We should show that 𝜑 (𝑓 ∞ , 𝜉 ′′ 𝑛 , 𝜎 ′′ 0 , 𝜎 ′′ 1 ) holds. By the definition of 𝑓 ∞ , there exists 𝑚 ∈ N such that Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

  𝑑𝜉 𝑛 prs □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑔 pvars □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ), vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 )

	Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference	12:47
	∫
	=

vals □ (𝑐 ′ 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

•

  The six equality comes from that 𝑝 𝑐 𝑔 𝜋 ,𝜎 𝜃 is a probability density by Remark 4.3.We define the partial density version of prs ⟨𝑆 ⟩ □ (𝑐) for 𝑆 ⊆ Name:𝜇 ∈dom(𝜉 𝑛 )∩𝑆 𝑐 (𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 ) (pr 𝜇 ) if ∃𝜎 𝑟 . used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ) 0 otherwise.prs ⟨𝑆 ⟩ □ (𝑐) enjoys many of the properties that prs □ (𝑐) has. For instance, prs ⟨𝑆 ⟩ □ (𝑐) is a well-defined function (i.e., its value does not depend on the choice of 𝜎 𝑟 ), as prs □ (𝑐) does. Since the proof of those properties of prs ⟨𝑆 ⟩ □ (𝑐) would be almost identical to that of prs □ (𝑐), we will use them in the following proofs without explicitly (re)proving them.Proof of Lemma D.3. Let 𝑐 be a command that has no observe commands. Let 𝑆 ⊆ Name, 𝜎 𝜃 ∈ St[𝜃 ], and 𝜎 𝑛 ∈ St[Name]. We set 𝜎 0 as in the definition of 𝑝 𝑐,𝜎 𝜃 in Eq. (3) (as a function of 𝜎 𝑛 ). Let 𝜎 = 𝜎 𝜃 ⊕ 𝜎 𝑛 ⊕ 𝜎 0 . If noerr (𝑐, 𝜎) does not hold, then the LHS and RHS of the desired equation become zero, so the equation holds. If noerr (𝑐, 𝜎) holds, we get the desired equation as follows:

	Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference	12:57
	D.3 Proofs of Lemmas D.3 and D.4
	prs ⟨𝑆 ⟩ □ (𝑐) : St[PVar] × St □ [Name] → [0, ∞),
	prs ⟨𝑆 ⟩ □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) ≜
	This completes the proof.	□

Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

  pr 𝜇 for some 𝜇 𝑓 val (𝜎 𝑛 (𝜇)) if 𝑣 ≡ val 𝜇 for some 𝜇 𝑓 cnt (𝜎 𝑛 (𝜇)) if 𝑣 ≡ cnt 𝜇 for some 𝜇 1 if 𝑣 ≡ like, where 𝑓 val (𝑟 ) ≜ 𝑟 , 𝑓 pr (𝑟 ) ≜ N (𝑟 ; 0, 1), and 𝑓 cnt (𝑟 ) ≜ 0. Define initial states 𝜎, 𝜎 ∈ St for 𝑝 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) and 𝑝 𝑐,𝜎 𝜃 (𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )), respectively, as𝜎 ≜ 𝜎 𝑝 ⊕ 𝜎 𝑛 ⊕ 𝑓 * (𝜎 𝑛 ), 𝜎 ≜ 𝜎 𝑝 ⊕ 𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) ⊕ 𝑓 * (𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )),where𝜎 𝑝 ≜ 𝜎 𝜃 ⊕ (𝜆𝑣 ∈ PVar \ 𝜃 . 0) ∈ St[PVar].Then, the assumption 𝑝 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) > 0 implies noerr (𝑐 𝜋 , 𝜎) by the definition of 𝑝. From this, 𝜎 (like) = 1, and the definition of used, there exists𝜉 𝑛 ∈ St □ [Name]such that used (𝑐 𝜋 , 𝜎, 𝜉 𝑛 ). From this, we have𝜎 = 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , used (𝑐 𝜋 , 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ),for some 𝜎 𝑟 . Next, let 𝜉 𝑛 ≜ vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ). can apply Lemma D.6 to used (𝑐 𝜋 , 𝜎, 𝜉 𝑛 ), since all the sample commands of 𝑐 have 𝜆𝑦.𝑦 in their third arguments (by assumption). The application of the lemma gives:∀𝜎 ′ 𝑟 ∈ St[Var \ (PVar ∪ dom(𝜉 𝑛 ))]. 𝜎 ′ 𝑟 (like) = 1 =⇒ used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 ′ 𝑟 , 𝜉 𝑛 ), prs ⟨𝑆 ⟩ □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) = prs ⟨𝑆 ⟩ □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ), where the for-all part comes from Lemma C.7.We now show two claims. The first claims is: there exists 𝜎 𝑟 such that𝜎 = 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ).By the definition of 𝜎, it suffices to show that 𝜉 𝑛 = 𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) | dom(𝜉 𝑛 ) . This indeed holds as follows: for any 𝜇 ∈ dom(𝜉 𝑛 ), 𝜉 𝑛 (𝜇) = vals □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) (𝜇) = 𝑐 𝜋 𝜎 (val 𝜇 ) = 𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) (𝜇), where the second equality uses used (𝑐 𝜋 , 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ). The second claim is: for all 𝜇 ∈ 𝑆 \ dom(𝜉 𝑛 ), 𝜎 (pr 𝜇 ) = 𝜎 (pr 𝜇 ). is the proof of the claim: 𝜎 (pr 𝜇 ) = 𝑓 pr (𝜎 (𝜇)) = 𝑓 pr (𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 ) (𝜇)) = 𝑓 pr ( 𝑐 𝜋 𝜎 (val 𝜇 )) = 𝑓 pr (𝜎 (val 𝜇)) = 𝑓 pr (𝜎 (𝜇)); and 𝜎 (pr 𝜇 ) = 𝑓 pr (𝜎 (𝜇)); here the second last equality in the first equation uses Lemma C.6-(2) with 𝜇 ∉ dom(𝜉 𝑛 ) and used (𝑐 𝜋 , 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ), and the last equality in the first equation uses 𝑓 val (𝑟 ) = 𝑟 . Based on the observations made so far, we show the desired equation as follows: 𝑐 𝜋 𝜎 (pr 𝜇 ) • 𝜇 ∈𝑆\dom(𝜉 𝑛 ) 𝑐 𝜋 𝜎 (pr 𝜇 ) = prs ⟨𝑆 ⟩ □ (𝑐 𝜋 ) (𝜎 𝑝 , 𝜉 𝑛 ) • 𝑐,𝜎 𝜃 (𝑣 𝑐 𝜋 ,𝜎 𝜃 (𝜎 𝑛 )). first and last equalities are by the definition of 𝑝. The second equality uses used (𝑐 𝜋 , 𝜎 𝑝 ⊕𝜉 𝑛 ⊕𝜎 𝑟 , 𝜉 𝑛 ) and Lemma C.6-(2) with 𝜇 ∉ dom(𝜉 𝑛 ). The third equality uses dom(𝜉 𝑛 ) = dom(𝜉 𝑛 ), the observation made in the first paragraph, and the second claim in the above. The fourth equality uses the first claim in the above, and Lemma C.6-(2) with 𝜇 ∉ dom(𝜉 𝑛 ). □ Lemma D.5. Let 𝑐 be a command and 𝜎 ∈ St. If 𝑐 has no observe commands and 𝑐 𝜎 ∈ St, then 𝑐 𝜎 (like) = 𝜎 (like).

	We Here 𝑝 ⟨𝑆 ⟩ 𝑐 𝜋 ,𝜎 𝜃	(𝜎 𝑛 ) =
		𝜇 ∈𝑆∩dom(𝜉 𝑛 )
				𝜎 (pr 𝜇 )
			𝜇 ∈𝑆\dom(𝜉 𝑛 )
		= prs ⟨𝑆 ⟩ □ (𝑐) (𝜎 𝑝 , 𝜉 𝑛 ) •	𝜎 (pr 𝜇 )
			𝜇 ∈𝑆\dom(𝜉 𝑛 )
		=	𝑐 𝜎 (pr 𝜇 ) •	𝑐 𝜎 (pr 𝜇 )
		𝜇 ∈𝑆∩dom(𝜉 𝑛 )	𝜇 ∈𝑆\dom(𝜉 𝑛 )
		= 𝑝 ⟨𝑆 ⟩
	The

  𝑛where the third equality uses used (𝑐 ′ 𝜋 , 𝜎, 𝜉 ′ 𝑛 ), used (𝑐 ′ 𝜋 ; 𝑐 ′′ 𝜋 , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), and the second claim in the proof of Lemma C.9.We now show the first conclusion. By IH on (𝑐 ′ , 𝜎 𝑝 , 𝜉 ′ 𝑛 ) and (𝑐 ′′ , 𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 ), we get ∃𝜎 ′ 𝑟 . used (𝑐 ′ , 𝜎 𝑝 ⊕ 𝜉 ′ 𝑐 ′ 𝜎 ∈ St by used (𝑐 ′ , 𝜎 𝑝 ⊕ 𝜉 ′ 𝑛 ⊕ 𝜎 ′ 𝑟 , 𝜉 ′ 𝑛 ) and Lemma C.6-(1), and we have 𝜎 | PVar = 𝜎 𝑝 , ( 𝑐 ′ 𝜎)| PVar = ( 𝑐 ′ (𝜎 𝑝 ⊕ 𝜉 ′ By these, used (𝑐 ′ , 𝜎 𝑝 ⊕ 𝜉 ′ 𝑛 ⊕ 𝜎 ′ 𝑟 , 𝜉 ′ 𝑛 ), used (𝑐 ′′ , 𝜎 ′ 𝑝 ⊕ 𝜉 ′′ 𝑛 ⊕ 𝜎 ′′ 𝑟 , 𝜉 ′′ 𝑛 ), and Lemma C.7, we get used (𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ), used -(𝑐 ′′ , 𝑐 ′ 𝜎, 𝜉 ′′ 𝑛 ). By these and the third claim in the proof of Lemma C.9, we get used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ). By this and Lemma C.7, we get the following as desired, since 𝜉 𝑛 = 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 (shown in the above) and 𝜎 = 𝜎 𝑝 ⊕ (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) ⊕ 𝜎 𝑟 for some 𝜎 𝑟 : used (𝑐, 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 , 𝜉 𝑛 ).

	= 𝜉 ′ 𝑛 ⊕ 𝜉 ′′
		𝑛 ⊕ 𝜎 ′ 𝑟 , 𝜉 ′ 𝑛 ),	∃𝜎 ′′ 𝑟 . used (𝑐 ′′ , 𝜎 ′ 𝑝 ⊕ 𝜉 ′′ 𝑛 ⊕ 𝜎 ′′ 𝑟 , 𝜉 ′′ 𝑛 ).
	Let
		𝜎 ≜ 𝜎 𝑝 ⊕ (𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ) ⊕ 𝜎 ′ 𝑟 | dom(𝜎 ′ 𝑟 )\dom(𝜉 ′′ 𝑛 ) .
	Then, 𝑛 ⊕ 𝜎 ′ 𝑟 ))| PVar	[By Lemma C.6-(3)]
		= pvars(𝑐 ′ ) (𝜎 𝑝 , 𝜉 ′ 𝑛 ) = 𝜎 ′ 𝑝 ,	[By the above]
	𝜎 | dom(𝜉 ′ 𝑛 ) = 𝜉 ′ 𝑛 ,	( 𝑐 ′ 𝜎)| dom(𝜉 ′′ 𝑛 ) = 𝜎 | dom(𝜉 ′′ 𝑛 ) = 𝜉 ′′ 𝑛 .	[By Lemma C.4]
		′′ 𝜋 ) (𝜎 ′ 𝑝 , 𝜉 ′′ 𝑛 )		[By the former equation]

  ′ 𝜋 , 𝜎, 𝜉 ′ 𝑛 ), used (𝑐 ′ 𝜋 ; 𝑐 ′′ 𝜋 , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ), used (𝑐 ′ , 𝜎, 𝜉 ′ 𝑛 ), and used (𝑐 ′ ; 𝑐 ′′ , 𝜎, 𝜉 ′ 𝑛 ⊕ 𝜉 ′′ 𝑛 ).

  𝑔(𝜃 𝑖 ) = ∫ R 𝑛 ℎ 𝑖 (𝑥) 𝑑𝑥 by the definition of ℎ 𝑖 , and lim 𝑖→∞ ℎ 𝑖 (𝑥) = (∇ 𝜃 𝑓 (𝜃, 𝑥))| 𝜃 =𝜃 ′ for almost all 𝑥 ∈ R 𝑛 by the second condition of this theorem. So, if the dominated convergence theorem is applicable to lim 𝑖→∞ ∫ R 𝑛 ℎ 𝑖 (𝑥) 𝑑𝑥, we would have lim

To do so, consider

{𝜃 𝑖 } 𝑖 ∈N ⊆ R \ {𝜃 ′ } such that lim 𝑖→∞ 𝜃 𝑖 = 𝜃 ′ . Define ℎ 𝑖 : R 𝑛 → R as ℎ 𝑖 (𝑥) ≜ 𝑓 (𝜃 𝑖 , 𝑥) -𝑓 (𝜃 ′ , 𝑥) 𝜃 𝑖 -𝜃 ′ .

Then, 𝑖→∞ 𝑔(𝜃 𝑖 ) = lim 𝑖→∞

  • "There exist 𝐻 : R 𝑛 → R ∪ {∞} and 𝐼 ∈ N such that (a) ∫ R 𝑛 𝐻 (𝑥) 𝑑𝑥 is finite, and (b) for all 𝑖 ≥ 𝐼 , |ℎ 𝑖 (𝑥)| ≤ 𝐻 (𝑥) for almost all 𝑥 ∈ R 𝑛 ": By the fourth condition of this theorem, there is an open 𝑈 ⊆ R such that 𝜃 ′ ∈ 𝑈 and ∫ R 𝑈 ) 𝑑𝑥 is finite. Further, (b) holds as follows: by lim 𝑖→∞ 𝜃 𝑖 = 𝜃 ′ and 𝑈 being an open neighborhood of 𝜃 ′ , there is 𝐼 ∈ N such that 𝜃 𝑖 ∈ 𝑈 for all 𝑖 ≥ 𝐼 ; therefore, for all 𝑖 ≥ 𝐼 , |ℎ 𝑖

𝑛 Lip(𝑓 (-, 𝑥)| 𝑈 ) 𝑑𝑥 is finite. Let 𝐻 : R 𝑛 → R ∪ {∞} be 𝐻 (𝑥) ≜ Lip 𝑓 (-, 𝑥)| 𝑈 .

Then, (a) clearly holds since ∫ R 𝑛 Lip(𝑓 (-, 𝑥)|

  Lebesgue measure zero. Note that the differentiability of 𝑔 is importantly used here; if 𝑔 is nondifferentiable even at a point, 𝑈 can have positive measure. The chain rule for 𝑔 • 𝑓 holds for each 𝑥 ∈ 𝑈 and this concludes the proof.Proof of Theorem 4.6. The proof is essentially the same as the proof of Theorem 4.5, except that we invoke the following properties of local Lipschitzness (instead of differentiability): the composition of locally Lipschitz functions is again locally Lipschitz, and the differentiation rules for +, ×, and 𝑙𝑜𝑔 hold almost everywhere for locally Lipschitz functions (Lemma E.1).Proof of Theorem 5.6. We prove the theorem by induction on the structure of 𝑐. Let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯

	□
	E.2 Proof of Theorem 4.6
	□
	F DEFERRED RESULTS IN §5.2
	F.1 Proof of Theorem 5.6

Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

has

  𝑐 . By Theorem F.1 on 𝑐 ′ , we have |= Δ( 𝑐 ′ , 𝑉 ′ , ∅) and |= Δ( 𝑐 ′ , 𝑑 ′ (𝑤), {𝑤 }) for all 𝑤 ∈ Var. If 𝑝 ′′ (𝑣) 𝑐 = ∅, then |= Δ( 𝑐 ′ , 𝑉 ′ , 𝑝 ′′ (𝑣) 𝑐 ) holds, and if 𝑝 ′′ (𝑣) 𝑐 ≠ ∅, then |= Δ( 𝑐 ′ , (𝑑 ′ ) ∪ (𝑝 ′′ (𝑣) 𝑐 ), 𝑝 ′′ (𝑣) 𝑐 ) holds by the merging lemma for Δ (Lemma F.5). By Eq. (23) and the weakening lemma for Δ (Lemma F.3), we obtain (c) for both cases. Note that we crucially used 𝑝 (𝑣) 𝑐 ⊇ 𝑉 ′ to handle the case 𝑝 ′′ (𝑣) 𝑐 = ∅.Case 𝑐 ≡ (if 𝑏 {𝑐 ′ }, else {𝑐 ′′ }). Let (𝑝 ′ , 𝑑 ′ , 𝑉 ′ ) ≜ 𝑐 ′ ♯and (𝑝 ′′ , 𝑑 ′′ , 𝑉 ′′ ) ≜ 𝑐 ′′ ♯ . Then,

	𝑝

  Smoothness Analysis for Probabilistic Programs with Application to Optimised Variational Inference

	12:73

Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

  ) holds. For 𝑛 > 0, we have 𝑡 ′ Lemma F.6 (Merging; Φ). Let 𝑓 ∈ [St → St ⊥ ] and 𝐾, 𝐾 ′ , 𝐿, 𝐿 ′ ⊆ Var. Then, |= Φ(𝑓 , 𝐾, 𝐿) ∧ |= Φ(𝑓 , 𝐾 ′ , 𝐿 ′ ) =⇒ |= Φ(𝑓 , 𝐾 ∩ 𝐾 ′ , 𝐿 ∪ 𝐿 ′ ).Proof. Uses the weakening lemma for Φ (Lemma F.4), we have|= Φ(𝑓 , 𝐾 ∩ 𝐾 ′ , 𝐿) and |= Φ(𝑓 , 𝐾 ∩ 𝐾 ′ , 𝐿 ′ ).This and Assumption 4 then imply the desired conclusion. Concretely, for all 𝜏 ∈ St[(𝐾 ∩ 𝐾 ′ ) 𝑐 ], if 𝑔, 𝑔 1 , and 𝑔 2 are the following partial functions𝑔 : St[𝐾 ∩ 𝐾 ′ ] ⇀ St[𝐿 ∪ 𝐿 ′ ], 𝑔(𝜎 ′ ) ≜ (𝜋 Var,𝐿∪𝐿 ′ • 𝑓 ) (𝜎 ′ ⊕ 𝜏) if 𝑓 (𝜎 ′ ⊕ 𝜏) ∈ St 𝑔 1 : St[𝐾 ∩ 𝐾 ′ ] ⇀ St[𝐿], 𝑔 1 (𝜎 ′ ) ≜ (𝜋 Var,𝐿 • 𝑓 ) (𝜎 ′ ⊕ 𝜏) if 𝑓 (𝜎 ′ ⊕ 𝜏) ∈ St 𝑔 2 : St[𝐾 ∩ 𝐾 ′ ] ⇀ St[𝐿 ′ ], 𝑔 2 (𝜎 ′ ) ≜ (𝜋 Var,𝐿 ′ • 𝑓 ) (𝜎 ′ ⊕ 𝜏) if 𝑓 (𝜎 ′ ⊕ 𝜏) ∈ St undefined otherwise, then 𝑔 1 ∈ 𝜙 𝐾∩𝐾 ′ ,𝐿 , 𝑔 2 ∈ 𝜙 𝐾∩𝐾 ′ ,𝐿 ′ , and 𝑔 = ⟨𝑔 1 , 𝑔 2 ⟩, so that by Assumption 4, we have 𝑔 ∈ 𝜙 𝐾∩𝐾 ′ ,𝐿∪𝐿 ′ as desired. □ Lemma F.7 (Seqence). Let 𝑓 , 𝑔 ∈ [St → St ⊥ ] and 𝐾, 𝐿, 𝐿 ′ , 𝑀 ⊆ Var. Then, |= Φ(𝑓 , 𝐾, 𝐿) ∧ |= Φ(𝑔, 𝐿, 𝑀) ∧ |= Δ(𝑓 , 𝐾 𝑐 , 𝐿 ′ \ 𝐿) ∧ |= Δ(𝑔, 𝐿 ′ , 𝑀) =⇒ |= Φ(seq(𝑓 , 𝑔), 𝐾, 𝑀). Proof. Consider 𝑓 , 𝑔 ∈ [St → St ⊥ ] and 𝐾, 𝐿, 𝐿 ′ , 𝑀 ⊆ Var that satisfy the given conditions: |= Φ(𝑓 , 𝐾, 𝐿), |= Φ(𝑔, 𝐿, 𝑀), |= Δ(𝑓 , 𝐾 𝑐 , 𝐿 ′ \ 𝐿), and |= Δ(𝑔, 𝐿 ′ , 𝑀).To prove the conclusion, pick an arbitrary 𝜏 ∈ St[𝐾 𝑐 ]. We have to show ℎ ∈ 𝜙 𝐾,𝑀 , whereℎ(𝜎) = 𝜋 Var,𝑀 ((𝑔 † • 𝑓 ) (𝜎 ⊕ 𝜏)) if (𝑔 † • 𝑓 ) (𝜎 ⊕ 𝜏) ∈ St undefined otherwise.Observe that since |= Φ(𝑓 , 𝐾, 𝐿) and |= Φ(𝑔, 𝐿, 𝑀), we have ℎ 1 ∈ 𝜙 𝐾,𝐿 and ℎ 2 ∈ 𝜙 𝐿,𝑀 for any 𝜏 1 ∈ St[𝐾 𝑐 ] and 𝜏 2 ∈ St[𝐿 𝑐 ], where ℎ 1 and ℎ 2 are parameterised by 𝜏 1 and 𝜏 2 , and defined by ℎ 1 : St[𝐾] ⇀ St[𝐿], ℎ 1 (𝜎) ≜ 𝜋 Var,𝐿 (𝑓 (𝜎 ⊕ 𝜏 1 )) if 𝑓 (𝜎 ⊕ 𝜏 1 ) ∈ St undefined otherwise, ℎ 2 : St[𝐿] ⇀ St[𝑀], ℎ 2 (𝜎) ≜ 𝜋 Var,𝑀 (𝑔(𝜎 ⊕ 𝜏 2 )) if 𝑔(𝜎 ⊕ 𝜏 2 ) ∈ St undefined otherwise.

	undefined	otherwise,
	undefined	otherwise,

𝑛 (𝜎) = 𝐹 (𝑡 ′ 𝑛-1 ) (𝜎) = 𝜎 if 𝑏 𝜎 = false (𝑡 ′ 𝑛-1 ) † ( 𝑐 0 𝜎) if 𝑏 𝜎 = true

Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

Our semantics is an instance (or variant) of existing density semantics (e.g.,[START_REF] Lee | Towards verified stochastic variational inference for probabilistic programs[END_REF]), and is different from sampling

There are multiple valid ways to convert a string-real pair to a name (i.e., to define create_name); we choose just one of them.Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

In practice, one more assumption is required: the set of random variables sampled from the model should be the same as that from the guide. This assumption can be checked automatically, e.g., by[START_REF] Lee | Towards verified stochastic variational inference for probabilistic programs[END_REF][START_REF] Lew | Trace types and denotational semantics for sound programmable inference in probabilistic languages[END_REF]. In this work, however, this assumption is always satisfied: all random variables in Name are sampled by a sample command or at the beginning (via initialisation).Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

Conditional differentiability extends partial differentiability in the sense that the latter can be expressed as a conjunction of the former (but not vice versa): 𝑓 is partially differentiable in 𝐾 if and only if 𝑓 is conditionally differentiable in {𝑣 } for all 𝑣 ∈ 𝐾.Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

To be precise, our analysis does not require the admissibility assumption, not because our abstract domain is finite, but because given a loop, our analysis finds an inductive predicate that is "sufficiently admissible" in the sense that it is closed under the least upper bound of any chain that matters for soundness: the chain should be definable by some loop. More details are in §F.4.Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

Our implementation is available at https://github.com/wonyeol/smoothness-analysis and [Lee et al. 2022a]. Proc. ACM Program. Lang., Vol. 7, No. POPL, Article 12. Publication date: January 2023.

ACKNOWLEDGMENTS

We thank Hangyeol Yu for helping us prove Theorems 4.2 and 4.5, and anonymous reviewers for giving constructive comments. Lee was supported by Samsung Scholarship. Yang was supported by the Engineering Research Center Program through the National Research Foundation of Korea (NRF) funded by the Korean Government MSIT (NRF-2018R1A5A1059921) and also by the Institute for Basic Science (IBS-R029-C1). Rival was supported by the French ANR VeriAMOS project.

Case 𝑐 ≡ (if 𝑏 {𝑐 ′ } else {𝑐 ′′ }). In this case, 𝑐 𝜋 ≡ (if 𝑏 {𝑐 ′ 𝜋 } else {𝑐 ′′ 𝜋 }). Since fv(𝑏) ⊆ PVar, 𝑏 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) is constant for all 𝜎 𝑟 ∈ St [Var \ PVar]. Without loss of generality, assume 𝑏 (𝜎 𝑝 ⊕ 𝜎 𝑟 ) = true. Then, 𝑐 (𝜎 𝑝 ⊕𝜎 𝑟 ) = 𝑐 ′ (𝜎 𝑝 ⊕𝜎 𝑟 ) and 𝑐 𝜋 (𝜎 𝑝 ⊕𝜎 𝑟 ) = 𝑐 ′ 𝜋 (𝜎 𝑝 ⊕𝜎 𝑟 ) for all 𝜎 𝑟 ∈ St [Var\PVar].

Hence, by IH on (𝑐 ′ , 𝜎 𝑝 , 𝜉 𝑛 ), we get the three conclusions directly.

Case 𝑐 ≡ (while 𝑏 {𝑐 ′ }). In this case, 𝑐 𝜋 ≡ (while 𝑏 {𝑐 ′ 𝜋 }). Consider the version of prs □ (-) where the parameter can be a state transformer 𝑓 : St → St ⊥ , instead of a command. Similarly, consider the version of the three conclusions where we use two state transformers 𝑓 , 𝑓 : St → St ⊥ , again instead of a command. We denote the versions by prs □ (𝑓 ) and 𝜑 (𝑓 , 𝑓 , 𝜎 𝑝 , 𝜉 𝑛 ). We write 𝑓 ∼ 𝑓 if prs □ (𝑓 ) (𝜎 𝑝 , 𝜉 𝑛 ) > 0 implies 𝜑 (𝑓 , 𝑓 , 𝜎 𝑝 , 𝜉 𝑛 ) for all 𝜎 𝑝 ∈ St [PVar] and 𝜉 𝑛 ∈ St □ [Name]. Further, we define

as

Note that 𝐹 𝜋 and 𝐹 𝜋 0 are the operators used in the semantics of the loops 𝑐 𝜋 and 𝑐 , respectively, where 𝜋 0 denotes the empty reparameterisation plan. We will show three claims: 𝜆𝜎.⊥ ∼ 𝜆𝜎.⊥; if 𝑓 ∼ 𝑓 , then 𝐹 𝜋 (𝑓 ) ∼ 𝐹 𝜋 0 (𝑓 ); and if increasing sequences {𝑓 𝑘 } 𝑘 ∈N and {𝑓 𝑘 } 𝑘 ∈N satisfy 𝑓 𝑘 ∼ 𝑓 𝑘 for all 𝑘 ∈ N, then 𝑓 ∞ ∼ 𝑓 ∞ holds for 𝑓 ∞ = 𝑘 ∈N 𝑓 𝑘 and 𝑓 ∞ = 𝑘 ∈N 𝑓 𝑘 . These three claims imply 𝑐 𝜋 ∼ 𝑐 , which in turn proves the desired three conclusions.

The first claim holds simply because prs □ (𝜆𝜎.⊥) (-, -) is always 0. To show the second claim, consider 𝑓 , 𝑓 : St → St ⊥ such that 𝑓 ∼ 𝑓 . We first replay our proof for the sequential-composition case on (𝑓 , 𝑓 ) after viewing 𝑓 † • 𝑐 ′ 𝜋 and 𝑓 † • 𝑐 ′ as the sequential composition of 𝑐 ′ 𝜋 and 𝑓 , and of 𝑐 ′ and 𝑓 , respectively. This replay, then, gives the relationship 𝑓 † • 𝑐 ′ 𝜋 ∼ 𝑓 † • 𝑐 ′ . Next, we replay our proof for the if case on (𝐹 𝜋 (𝑓 ), 𝐹 𝜋 0 (𝑓 )), after viewing 𝑓 † • 𝑐 ′ 𝜋 and 𝑓 † • 𝑐 ′ as the true branches, and 𝜆𝜎. 𝜎 = skip as the false branch. This replay implies the required relationship 𝐹 𝜋 (𝑓 ) ∼ 𝐹 𝜋 0 (𝑓 ).

To show the third condition, consider increasing sequences {𝑓 𝑘 } 𝑘 ∈N and {𝑓 𝑘 } 𝑘 ∈N such that 𝑓 𝑘 ∼ 𝑓 𝑘 for all 𝑘 ∈ N. Let 𝑓 ∞ = 𝑘 ∈N 𝑓 𝑘 and 𝑓 ∞ = 𝑘 ∈N 𝑓 𝑘 . Consider any 𝜎 𝑝 and 𝜉 𝑛 such that prs □ (𝑓 ∞ ) (𝜎 𝑝 , 𝜉 𝑛 ) > 0. We should show 𝜑 (𝑓 ∞ , 𝑓 ∞ , 𝜎 𝑝 , 𝜉 𝑛 ). Pick any 𝜎 𝑟 ∈ St[Var \ (PVar ∪ dom(𝜉 𝑛 ))] with 𝜎 𝑟 (like) = 1. Let 𝜎 = 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 and 𝜎 = 𝜎 𝑝 ⊕ 𝜉 𝑛 ⊕ 𝜎 𝑟 . Note that the value of each term in

) is independent of the choice of 𝜎 𝑟 by Lemma C.7 and the well-definedness of pvars □ and prs ⟨𝑆 ⟩ □ . Since the two given sequences are increasing, there exists 𝐾 ∈ N such that 𝑓 ∞ (𝜎) = 𝑓 𝐾 (𝜎) and 𝑓 ∞ (𝜎) = 𝑓 𝐾 (𝜎). From this and prs □ (𝑓 ∞ ) (𝜎 𝑝 , 𝜉 𝑛 ) > 0, we have prs □ (𝑓 𝐾 ) (𝜎 𝑝 , 𝜉 𝑛 ) > 0. This in turn gives 𝜑 (𝑓 𝐾 , 𝑓 𝐾 , 𝜎 𝑝 , 𝜉 𝑛 ) since 𝑓 𝐾 ∼ 𝑓 𝐾 . Lastly, again by

Deferred Statements and Their Proofs

Lemma E.1. Let 𝑓 , 𝑔 : R 𝑛 → R be locally Lipschitz functions. Then, the following differentiation rules hold for almost every 𝑥 ∈ R 𝑛 :

where for the third rule we assume 𝑓 (𝑦) > 0 for all 𝑦 ∈ R 𝑛 .

Our program analysis consists of two parts, one for tracking the dependency information and the other for tracking the smoothness information. The first part does not depend on the second, although it is used crucially by the second part. We exploit this one-way relationship between the two parts of our analysis, and prove the soundness of the dependency-tracking part first and then that of the other smoothness-tracking part. Consider a command 𝑐, and let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯ . Then, we have:

Theorem F.1. For all 𝑣 ∈ Var, we have |= Δ( 𝑐 , 𝑑 (𝑣), {𝑣 }). Also, |= Δ( 𝑐 , 𝑉 , ∅).

Theorem F.2. For all 𝑣 ∈ Var, we have |= Φ( 𝑐 , 𝑝 (𝑣), {𝑣 }).

We prove the two soundness results in §F.3 and §F.4. From these, we immediately obtain the main soundness theorem:

Proof of Theorem 5.8. Let 𝑐 be a command and (𝑝, 𝑑, 𝑉 ) = 𝑐 ♯ . Then, by Theorems F.1 and F.2, we have |= Δ( 𝑐 , 𝑑 (𝑣), {𝑣 }), |= Δ( 𝑐 , 𝑉 , ∅), and |= Φ( 𝑐 , 𝑝 (𝑣), {𝑣 }) for all 𝑣 ∈ Var. Hence, by the definition of 𝛾 (i.e., Eq. ( 10)), we have 𝑐 ∈ 𝛾 ( 𝑐 ♯ ) as desired.

□

Proof of Theorem F.1. We prove the theorem by induction on the structure of 𝑐. Let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯

. Pick a variable 𝑣 ∈ Var and states 𝜎, 𝜎 ′ , 𝜎 0 , 𝜎 ′ 0 ∈ St such that 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ and 𝜎 0 ∼ 𝑉 𝜎 ′ 0 . We will show that (i) if 𝑐 𝜎 0 ∈ St, then 𝑐 𝜎 ′ 0 ∈ St, and (ii) if 𝑐 𝜎 ∈ St and 𝑐 𝜎 ′ ∈ St, then 𝑐 𝜎 ∼ {𝑣 } 𝑐 𝜎 ′ , i.e., 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣). Since 𝑉 ⊆ 𝑑 (𝑣), these two imply the claim of the theorem. We refer to these two properties as conditions (i) and (ii) in the rest of the proof.

Case 𝑐 ≡ skip. In this case, 𝑑 (𝑣) = {𝑣 } and 𝑉 = ∅. The condition (i) holds since skip always terminates. The condition (ii) also holds because 𝑐 𝜎 ′′ = 𝜎 ′′ for all 𝜎 ′′ , and the relation ∼ 𝑑 (𝑣) coincides with ∼ {𝑣 } .

Case 𝑐 ≡ (𝑥 := 𝑒). In this case, 𝑉 = ∅, and the condition (i) holds since the assignments always terminate. For the condition (ii), we do case analysis on the variable 𝑣.

• Case 𝑣 ≡ 𝑥. In this case, 𝑑 (𝑣) = fv(𝑒). This implies 𝑒 𝜎 = 𝑒 𝜎 ′ . Thus, 𝑐 𝜎 (𝑥) = 𝑒 𝜎 = 𝑒 𝜎 ′ = 𝑐 𝜎 ′ (𝑥). This implies the desired 𝑐 𝜎 ∼ {𝑥 } 𝑐 𝜎 ′ . • Case 𝑣 𝑥. In this case, 𝑑 (𝑣) = {𝑣 }, and so 𝜎 (𝑣) = 𝜎 ′ (𝑣). This implies that 𝑐 𝜎 (𝑣) = 𝜎 (𝑣) = 𝜎 ′ (𝑣) = 𝑐 𝜎 ′ (𝑣), which gives the desired relationship.

Case 𝑐 ≡ obs(dist N (𝑒 1 , 𝑒 2 ), 𝑟 ). The observe commands always terminate. Thus, the condition (i) holds. We prove the condition (ii) by case analysis on the variable 𝑣. If 𝑣 is not like, then 𝑑 (𝑣) = {𝑣 }, 𝑐 𝜎 (𝑣) = 𝜎 (𝑣), and 𝑐 𝜎 ′ (𝑣) = 𝜎 ′ (𝑣). Thus, in this case, the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ implies 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣), as desired. If 𝑣 is like, then 𝑑 (𝑣) = fv(𝑒 1 ) ∪ fv(𝑒 2 ) ∪ {like}, and for some function 𝑔 : R 4 → R, 𝑐 𝜎 (𝑣) = 𝑔(𝜎 (like), 𝑟, 𝑒 1 𝜎, 𝑒 2 𝜎) and 𝑐 𝜎 ′ (𝑣) = 𝑔(𝜎 ′ (like), 𝑟, 𝑒 1 𝜎 ′ , 𝑒 2 𝜎 ′ ).

Therefore, from the assumption 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ , it follows that 𝑐 𝜎 (𝑣) = 𝑐 𝜎 ′ (𝑣), as desired.

Next we show the condition (𝑖𝑖) under the assumption that 𝑏 𝜎 = true. As before, the proof of the other case 𝑏 𝜎 = false is essentially the same. Since 𝑑 (𝑣) includes fv(𝑏) and 𝑑 ′ (𝑣), we have 𝑏 𝜎 ′ = true and 𝜎 ∼ 𝑑 ′ (𝑣) 𝜎 ′ . Also, because 𝑐 𝜎 = 𝑐 ′ 𝜎 and 𝑐 𝜎 ′ = 𝑐 ′ 𝜎 ′ , both 𝑐 ′ 𝜎 and 𝑐 ′ 𝜎 ′ are in St. Thus, by induction hypothesis, 𝑐 ′ 𝜎 ∼ {𝑣 } 𝑐 ′ 𝜎 ′ , which implies that

as desired.

Case 𝑐 ≡ (while 𝑏 {𝑐 0 }). Let (𝑑 0 , 𝑝 0 , 𝑉 0 ) ≜ 𝑐 0

♯

, and 𝐹 ♯ be the operator in the abstract semantics of 𝑐 such that (𝑝, 𝑑, 𝑉 ) is the least fixed point of 𝐹 ♯ . Also, let 𝐹 be the operator in the concrete semantics of 𝑐 such that 𝑐 is the least fixed point of 𝐹 . Now define

We will show that (i) 𝑇 contains the empty function ⊥ St→St ⊥ ≜ 𝜆𝜎. undefined, (ii) it is closed under the least upper bounds of increasing chains, and (iii) the function 𝐹 maps functions in 𝑇 to some functions in the same set. These three imply that the least fixed point of 𝐹 , namely, 𝑐 , is in 𝑇 , which gives the desired conclusion.

The membership of

To prove the next requirement, namely, the closure under the least upper bounds of increasing chains, consider a chain (𝑓 𝑛 ) 𝑛 ∈N in 𝑇 , i.e., a sequence such that 𝑓 𝑛 (𝜎) = 𝑓 𝑛+1 (𝜎) for all 𝑛 ∈ N and

As in all the other cases so far, we pick an arbitrary variable 𝑣 ∈ Var and arbitrary states 𝜎 0 , 𝜎 ′ 0 , 𝜎, and

We will show that 𝑓 ∞ (𝜎 ′ 0 ) ∈ St and 𝑓 ∞ (𝜎) ∼ {𝑣 } 𝑓 ∞ (𝜎 ′ ), which correspond to what we have called conditions (i) and (ii) in the previous cases. Since 𝑓 ∞ (𝜎 0 ) ∈ St, there exists

Our proof of the condition (ii) has a similar form. Since both 𝑓 ∞ (𝜎) and 𝑓 ∞ (𝜎 ′ ) are in St, there exists 𝑛 ∈ N such that 𝑓 ∞ (𝜎) = 𝑓 𝑛 (𝜎) and 𝑓 ∞ (𝜎 ′ ) = 𝑓 𝑛 (𝜎 ′ ). By assumption, 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ , and 𝑓 𝑛 ∈ 𝑇 . Thus, 𝑓 𝑛 (𝜎) ∼ {𝑣 } 𝑓 𝑛 (𝜎 ′ ), which gives the desired 𝑓 ∞ (𝜎) ∼ {𝑣 } 𝑓 ∞ (𝜎 ′ ).

It remains to show the last requirement, i.e., the closure under 𝐹 . Pick an arbitrary 𝑓 ∈ 𝑇 . Consider a variable 𝑣 ∈ Var and states 𝜎 0 , 𝜎 ′ 0 , 𝜎, and

We will show that 𝐹 (𝑓 ) (𝜎 ′ 0 ) ∈ St and 𝐹 (𝑓 ) (𝜎) ∼ {𝑣 } 𝐹 (𝑓 ) (𝜎 ′ ), while referring to these two desired properties as conditions (i) and (ii), as we have done before.

Let us handle the condition (i) first. If 𝑏 𝜎 0 = false, we have

Next, we prove the condition (ii). If 𝑏 𝜎 = false, we have 𝑏 𝜎 ′ = false since fv(𝑏) ⊆ 𝑑 (𝑣) and 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ . Thus, in this case, 𝐹 (𝑓 ) (𝜎) = 𝜎 and 𝐹 (𝑓 ) (𝜎 ′ ) = 𝜎 ′ . Also, {𝑣 } ⊆ 𝑑 (𝑣), and so, 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ implies that 𝐹 (𝑓 ) (𝜎) = 𝜎 ∼ {𝑣 } 𝜎 ′ = 𝐹 (𝑓 ) (𝜎 ′ ), as desired. Now assume that 𝑏 𝜎 = true. Then, 𝑏 𝜎 ′ = true by the reason that fv(𝑏) ⊆ 𝑑 (𝑣) and 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ . Also, 𝑐 0 𝜎 and 𝑐 0 𝜎 ′ are in St, so that 𝐹 (𝑓 ) (𝜎) = 𝑓 ( 𝑐 0 𝜎) and 𝐹 (𝑓 ) (𝜎 ′ ) = 𝑓 ( 𝑐 0 𝜎 ′ ). Furthermore, since 𝑑 (𝑣) ⊇ (𝑑 0 ) ∪ (𝑑 (𝑣)) and 𝜎 ∼ 𝑑 (𝑣) 𝜎 ′ , we have 𝑐 0 𝜎 ∼ 𝑑 (𝑣) 𝑐 0 𝜎 ′ . We then use the fact that 𝑓 ∈ 𝑇 and 𝑓 ( 𝑐 0 𝜎), 𝑓 ( 𝑐 0 𝜎 ′ ) ∈ St, and conclude that 𝑓 ( 𝑐 0 𝜎) ∼ {𝑣 } 𝑓 ( 𝑐 0 𝜎 ′ ), which gives the desired 𝐹 (𝑓 ) (𝜎) ∼ {𝑣 } 𝐹 (𝑓 ) (𝜎 ′ ). □ F.4 Proof of Theorem F.2

Let seq be the following operator, which models sequential composition:

Also, define an operator cond for modelling if commands as follows:

cond :

Proof of Theorem F.2. We prove the theorem by induction on the structure of𝑐. Let (𝑝, 𝑑, 𝑉 ) ≜ 𝑐 ♯ .

Case 𝑐 ≡ skip. In this case, 𝑐 (𝜎) = 𝜎 for all 𝜎 ∈ St, and 𝑝 (𝑣) = Var for all 𝑣 ∈ Var. To prove the conclusion, consider 𝑣 ∈ Var and

The second equality is by the definition of 𝐹 , the third by induction hypothesis, and the last by the following: for any 𝜎 ∈ St and 𝑗 ∈ {1, 2}, ( * 𝑗 ) holds iff ( * ′ 𝑗 ) holds; and for any 𝑖 ∈ N, 𝑐 0 𝜎 ∈ St implies 𝑐 (𝑖) 0 ( 𝑐 0 𝜎) = 𝑐 (𝑖+1) 0 𝜎, and 𝑐 (𝑖+1)

Hence, Eq. ( 26) holds. This completes the proof of (iii).

Proof of (iv). To prove this, we prove a stronger statement: for all 𝑛 ∈ N, |= Δ( 𝑐 (𝑛) 0 , 𝑉 ′ 𝑛+1 , fv(𝑏)). This statement implies (iv) since 𝑉 ′ 𝑛 ⊆ 𝑉 ′ ∞ for all 𝑛 ∈ N and we have the weakening lemma for Δ (Lemma F.3). We prove the statement by induction on 𝑛. For 𝑛 = 0, 𝑐 (𝑛) 0 = skip and 𝑉 ′ 𝑛+1 = fv(𝑏) ∪ 𝑉 0 . By Theorem F.1 on skip, we have |= Δ( skip , {𝑣 }, {𝑣 }) for all 𝑣 ∈ Var, and then by the merging lemma for Δ (Lemma F.5), we have |= Δ( skip , fv(𝑏), fv(𝑏)).

holds by the weakening lemma for Δ (Lemma F.3). Next, for 𝑛 > 0, 𝑐 (𝑛)

on 𝑐 0 , and induction hypothesis of the theorem (not that of the claim (iv)), we have

Also, by induction hypothesis of our strengthening of the claim (iv) and the weakening lemma for Δ (Lemma F.3), |= Δ( 𝑐 (𝑛-1) 0 , 𝑉 ′ 𝑛 , {𝑣 }) for all 𝑣 ∈ fv(𝑏). By applying to these the proof of Theorem F.1 (on the sequential composition case), we have |= Δ( 𝑐 0 ; 𝑐 (𝑛-1) 0 , 𝑉 0 ∪ (𝑑 0 ) ∪ (𝑉 ′ 𝑛 ), {𝑣 }) for all 𝑣 ∈ fv(𝑏). By the merging lemma for Δ (Lemma F.5), |= Δ( 𝑐 (𝑛)

) by the weakening lemma for Δ (Lemma F.3). This completes the proof of (iv).

Case 𝑐 ≡ (𝑥 := sam(name(𝛼, 𝑒), dist N (𝑒 1 , 𝑒 2 ), 𝜆𝑦.𝑒 ′ )). To prove the conclusion, consider 𝑣 ∈ Var and 𝜏 ∈ St[𝑝 (𝑣) 𝑐 ]. We should show 𝑔 ∈ 𝜙 𝑝 (𝑣),{𝑣 } , where

We prove this by case analysis on 𝑣. if 𝑒 is a constant, fv(𝑒) = ∅ holds, and if 𝑒 is not a constant, the definition of 𝑝 (𝑣) ensures this. Thus, there exists 𝜇 0 ∈ Name such that create_name(𝛼, 𝑒 (𝜎 ⊕ 𝜏)) = 𝜇 0 for all 𝜎 ∈ St[𝑝 (𝑣)]. We now do refined case analysis on 𝑣 using 𝜇 0 .

• Case 𝑣 ∈ {val 𝜇 , pr 𝜇 , cnt 𝜇 | 𝜇 ∈ Name, 𝜇 = (𝛼, _), 𝜇 ≠ 𝜇 0 }. In this case,

Here the first equality holds since 𝑐 does not change the value of 𝑣. By Assumption 3, we have 𝜋 Var,{𝑣 } ∈ 𝜙 Var,{𝑣 } . Then, by Assumption 5, we obtain 𝑔 ∈ 𝜙 𝑝 (𝑣),{𝑣 } . Note that this argument does not depend on the value of 𝑝 (𝑣) (which can be Var, fv(𝑒) 𝑐 ∩ 𝑣 + 1 ♯ , etc., depending on 𝑒 and 𝑣).

Then,

for all 𝜎 ∈ St[𝑝 (𝑣)]. By Assumption 2, we have ℎ ∈ 𝜙 𝐾 1 ,{𝑣 } , Then, by Assumption 5, we obtain 𝑔 ∈ 𝜙 𝑝 (𝑣),{𝑣 } .

• Case 𝑣 ≡ pr 𝜇 0 . In this case,

Using 𝜏 1 and 𝜏 2 , we have

Here the first equality follows from the definition of 𝑐 , the second equality holds because pdf N is the density function of a normal distribution, and the third equality comes from 𝜏 = 𝜏 1 ⊕ 𝜏 2 . By Assumption 2, we have ℎ ∈ 𝜙 𝐾,{𝑣 } . Then, by Assumption 5, we obtain 𝑔 ∈ 𝜙 𝑝 (𝑣),{𝑣 } . • Case 𝑣 ≡ cnt 𝜇 0 . The proof is similar to the above case 𝑣 ≡ pr 𝜇 0 . In this case, 𝑝 (𝑣) = fv(𝑒) 𝑐 ∩ 𝐾 for 𝐾 = cnt 𝜇 0 + 1 ♯ . As in the above case, there exist 𝜏 1 ∈ St[𝐾 \ fv(𝑒) 𝑐 ] and 𝜏 2 ∈ St[𝐾 𝑐 ] such that 𝜏 = 𝜏 1 ⊕ 𝜏 2 , and we have

By Assumption 2, we have ℎ ∈ 𝜙 𝐾,{𝑣 } . Then, by Assumption 5, we obtain 𝑔 ∈ 𝜙 𝑝 (𝑣),{𝑣 } .

Case 𝑐 ≡ obs(dist N (𝑒 1 , 𝑒 2 ), 𝑟 ). To prove the conclusion, consider 𝑣 ∈ Var and 𝜏 ∈ St[𝑝 (𝑣) 𝑐 ]. We should show 𝑔 ∈ 𝜙 𝑝 (𝑣),{𝑣 } , where

We prove this by case analysis on 𝑣. Next, suppose 𝑣 ≡ like. Then, 𝑝 (𝑣) = like × pdf N (𝑟 ; 𝑒 1 , 𝑒 2 ) ♯ and

Here the first equality is by the definition of 𝑐 , and the second equality holds because pdf N is the density function of a normal distribution. Hence, by Assumption 2, we have 𝑔 ∈ 𝜙 𝑝 (𝑣),{𝑣 } . □ F.5 Proofs of Lemmas for Theorem F.2

Here are the lemmas used to prove Theorem F.2:

Note that the conclusion of the second conjunct implies 𝑓 (𝜎) ∼ 𝐿 ′ 𝑓 (𝜎 ′ ) since 𝐿 ′ ⊆ 𝐿. From what we have just shown, the desired conclusion |= Δ(𝑓 , 𝐾 ′ , 𝐿 ′ ) follows.

Proof. We prove the lemma using Assumptions 3, 5 and 6. Consider 𝜏 ∈ St[(𝐾 ′ ) 𝑐 ], and let 𝑔 be the following partial function:

We should show𝑔 ∈ 𝜙 𝐾 ′ ,𝐿 ′ . Note that (𝐾 ′ ) 𝑐 ⊇ 𝐾 𝑐 . Thus, there exist𝜏

Then, since |= Φ(𝑓 , 𝐾, 𝐿), we have ℎ ∈ 𝜙 𝐾,𝐿 . Note that for all

By Assumptions 3, 5 and 6, the above equation implies 𝑔 ∈ 𝜙 𝐾 ′ ,𝐿 ′ , as desired.

Proof. Consider 𝜎, 𝜎 ′ ∈ St with 𝜎 ∼ 𝐾∪𝐾 ′ 𝜎 ′ . Then, 𝜎 ∼ 𝐾 𝜎 ′ , and by the assumption that |= Δ(𝑓 , 𝐾, 𝐿), we have This implies that 𝑓 (𝜎) ∼ 𝐿∪𝐿 ′ 𝑓 (𝜎 ′ ), as desired. □

H DEFERRED RESULTS IN §6

H.1 Proof of Theorem 6.2

Proof of Theorem 6.2. Suppose that the algorithm returns 𝜋 (without an error message). We should show the conclusion that 𝜋 is simple and satisfies (R2) and (R3).

We make several observations before proving the conclusion. Let

and 𝐾 ⊆ Var be the set defined in Eq. ( 12). Also, let 𝑆 𝑟 be the set of names that the algorithm uses to construct the returned reparameterisation plan 𝜋. Then, by the algorithm, we have the two inclusions in Eq. ( 12) and Eq. ( 13), and also

We now prove the conclusion in three parts.

First part: We show that 𝜋 is simple. To show this, consider (𝑛, 𝑑, 𝑙), (𝑛 ′ , 𝑑 ′ , 𝑙 ′ ) ∈ NameEx × DistEx×LamEx such that𝑛 = name(𝛼, 𝑒) and𝑛 ′ = name(𝛼, 𝑒 ′ ) for some 𝛼 ∈ Str,𝑒, and𝑒 ′ . Suppose that (𝑛, 𝑑, 𝑙) ∈ dom(𝜋). We should show (𝑛 ′ , 𝑑 ′ , 𝑙 ′ ) ∈ dom(𝜋). Since (𝑛, 𝑑, 𝑙) ∈ dom(𝜋) = dom(𝜋 0 [𝑆 𝑟 ]), we have (𝑛, 𝑑, 𝑙) ∈ dom(𝜋 0 ) and (𝛼, _) ∈ 𝑆 𝑟 . This implies that (𝑛 ′ , 𝑑 ′ , 𝑙 ′ ) ∈ dom(𝜋 0 ) because 𝜋 0 is simple. Since 𝑛 ′ = name(𝛼, _) and (𝛼, _) ∈ 𝑆 𝑟 , we have (𝑛 ′ , 𝑑 ′ , 𝑙 ′ ) ∈ dom(𝜋) as desired.

Second part: We show that 𝜋 satisfies (R2) in three steps. First step: We prove 𝜃 ∪ rv(𝜋) ⊆ 𝐾. To do so, observe that the 𝑆 in the algorithm always satisfies the following property:

(𝛼, 𝑖) ∈ 𝑆 =⇒ (𝛼, 𝑖 ′ ) ∈ 𝑆 for any (𝛼, 𝑖), (𝛼, 𝑖 ′ ) ∈ Name. where the second and third equalities use the definitions of rv(-) and 𝜋 0 [𝑆], respectively, and the last equality uses Eq. ( 28). Since rv(𝜋) ⊆ 𝑆 ⊆ 𝐾 and 𝜃 ⊆ 𝐾 by Eq. ( 12) (the inclusion of 𝜃 ), we get 𝜃 ∪ rv(𝜋) ⊆ 𝐾 as desired.

Second step: We prove that for all 𝑢 ∈ {like} ∪ {pr 𝜇 | 𝜇 ∈ Name} and 𝑣 ∈ {pr 𝜇 | 𝜇 ∈ Name}, the following functions (which are total since 𝑐 𝑚 and 𝑐 𝑔 always terminate) are differentiable with respect to the variables in 𝜃 ∪ rv(𝜋) jointly:

where

(30) Note that in 𝑓 𝑚 and 𝑓 𝑔 , the AVar part does not depend on the Name part (unlike in Eq. ( 29)). For the proof, pick arbitrary 𝑢 ∈ {like} ∪ {pr 𝜇 | 𝜇 ∈ Name} and 𝑣 ∈ {pr 𝜇 | 𝜇 ∈ Name}. Then, 𝜃 ∪ rv(𝜋) ⊆ 𝐾 ⊆ p 𝑚 (𝑢) ∩ p 𝑔 (𝑣), where the first inclusion is from the above result and the second from Eq. ( 12) (the definition of 𝐾). By the soundness of differentiability analysis (Theorem 5.8), |= Φ( 𝑐 𝑚 , p 𝑚 (𝑢), {𝑢}) and |= Φ( 𝑐 𝑔 , p 𝑔 (𝑣), {𝑣 }). From this, and by the weakening lemma of Φ with 𝜃 ∪ rv(𝜋) ⊆ p 𝑚 (𝑢) ∩ p 𝑔 (𝑣) (Lemma F.4), we have |= Φ( 𝑐 𝑚 , 𝜃 ∪ rv(𝜋), {𝑢}) and |= Φ( 𝑐 𝑔 , 𝜃 ∪ rv(𝜋), {𝑣 }). Hence, the functions in Eq. ( 30) are differentiable with respect to 𝜃 ∪ rv(𝜋) jointly as desired, by the definition of Φ ( §5.1) and the definition of "𝑓 : St[𝐿] → R for 𝐿 ⊆ Var is differentiable with respect to 𝐿 ′ ⊆ 𝐿 jointly" ( §4.2).

Second substep: We now prove that the claim of the second step follows from the first substep just proved. Pick any 𝑢 ∈ {like} ∪ {pr 𝜇 | 𝜇 ∈ Name}. We should show that the first function in Eq. ( 29) is differentiable with respect to 𝜃 ∪ rv(𝜋). Note that we should also show the same for the second function (for any 𝑣), but the proof is similar to the first function so we omit this case. To prove the claim for the first function, pick any 𝜉 )), where 𝑔 is extended to accept a substate in St □ [Name] and return a substate for the corresponding auxiliary part. Then, to prove the claim, it suffices to show that 𝑓 ′′ is differentiable (since 𝜉 ′ 𝑛,0 was chosen arbitrarily). We do case analysis on 𝑈

)) for all 𝜎 𝜃 and 𝜉 𝑛 ; since𝑔, ⊕, and proj are all differentiable (because𝑔 only uses projection and the density of the standard normal distribution), 𝑓 ′′ is differentiable. Hence, 𝑓 ′′ is differentiable in both cases, and this shows the claim of the second step.

Third step: We prove that 𝜋 satisfies (R2), i.e., the functions in (R2) are differentiable with respect to 𝜃 ∪ rv(𝜋) jointly. This holds because: 𝑐 𝑚 and 𝑐 𝑔 do not have a double-sampling error, so each function in (R2) is a multiplication of some of the functions in Eq. ( 29) (for different 𝑢 and 𝑣); the functions in Eq. ( 29) are differentiable with respect to 𝜃 ∪ rv(𝜋) jointly (by the above result); and multiplication preserves differentiability. 4). "Manual" and "Ours" denote the number of continuous random variables and learnable parameters in which the density of the program is smooth, computed by hand and by our analyser. "Time" denotes the runtime of our analyser in seconds. "#CRP" denotes the total number of continuous random variables and learnable parameters in the program. -m and -g denote model and guide. We consider {(𝛼, 𝑖) ∈ Name} as one random variable for each 𝛼 ∈ Name.

Differentiable

Locally 5). "Ours-Time" denote the runtime of our variable selector in seconds. "Ours-Sound" and "Pyro \ Ours" denote the number of random variables in the example that are in 𝜋 ours , and that are in 𝜋 0 but not in 𝜋 ours , respectively, where 𝜋 ours and 𝜋 0 denote the reparameterisation plans given by our variable selector and by Pyro. "Pyro \ Ours" is partitioned into "Sound" and "Unsound": the latter denotes the number of random variables that make (R2') or (R3') violated when added to 𝜋 ours , and the former denotes the number of the rest. "#CR" and "#DR" denote the total number of continuous and discrete random variables in the example. We consider {(𝛼, 𝑖) ∈ Name} as one random variable for each 𝛼 ∈ Name.

Ours