text:
Stochastic programming

In the field of mathematical optimization, stochastic programming is a framework for modeling optimization problems that involve uncertainty. A stochastic program is an optimization problem in which some or all problem parameters are uncertain, but follow known probability distributions. This framework contrasts with deterministic optimization, in which all problem parameters are assumed to be known exactly. The goal of stochastic programming is to find a decision which both optimizes some criteria chosen by the decision maker, and appropriately accounts for the uncertainty of the problem parameters. Because many real-world decisions involve uncertainty, stochastic programming has found applications in a broad range of areas ranging from finance to transportation to energy optimization.

Two-stage problems
The basic idea of two-stage stochastic programming is that (optimal) decisions should be based on data available at the time the decisions are made and cannot depend on future observations. The two-stage formulation is widely used in stochastic programming. The general formulation of a two-stage stochastic programming problem is given by: $$ \min_{x\in X}\{ g(x)= f(x) + E_{\xi}[Q(x,\xi)]\} $$ where $$Q(x,\xi)$$ is the optimal value of the second-stage problem $$ \min_{y}\{ q(y,\xi) \,|\,T(\xi)x+W(\xi) y = h(\xi)\}. $$

The classical two-stage linear stochastic programming problems can be formulated as $$ \begin{array}{llr} \min\limits_{x\in \mathbb{R}^n}  &g(x)= c^T x + E_{\xi}[Q(x,\xi)]    &   \\ \text{subject to} & Ax   =    b &\\ & x    \geq 0 & \end{array} $$

where $$ Q(x,\xi)$$ is the optimal value of the second-stage problem $$ \begin{array}{llr} \min\limits_{y\in \mathbb{R}^m}  & q(\xi)^T y     &   \\ \text{subject to} & T(\xi)x+W(\xi)y   =    h(\xi) &\\ & y    \geq 0 & \end{array} $$

In such formulation $$x\in \mathbb{R}^n$$ is the first-stage decision variable vector, $$y\in \mathbb{R}^m$$ is the second-stage decision variable vector, and $$\xi(q,T,W,h)$$ contains the data of the second-stage problem. In this formulation, at the first stage we have to make a "here-and-now" decision $$x$$ before the realization of the uncertain data $$\xi$$, viewed as a random vector, is known. At the second stage, after a realization of $$\xi$$ becomes available, we optimize our behavior by solving an appropriate optimization problem.

At the first stage we optimize (minimize in the above formulation) the cost $$c^Tx$$ of the first-stage decision plus the expected cost of the (optimal) second-stage decision. We can view the second-stage problem simply as an optimization problem which describes our supposedly optimal behavior when the uncertain data is revealed, or we can consider its solution as a recourse action where the term $$Wy$$ compensates for a possible inconsistency of the system $$Tx\leq h$$ and $$q^Ty$$ is the cost of this recourse action.

The considered two-stage problem is linear because the objective functions and the constraints are linear. Conceptually this is not essential and one can consider more general two-stage stochastic programs. For example, if the first-stage problem is integer, one could add integrality constraints to the first-stage problem so that the feasible set is discrete. Non-linear objectives and constraints could also be incorporated if needed.

Distributional assumption
The formulation of the above two-stage problem assumes that the second-stage data $$\xi$$ is modeled as a random vector with a known probability distribution. This would be justified in many situations. For example, the distribution of $$\xi$$ could be inferred from historical data if one assumes that the distribution does not significantly change over the considered period of time. Also, the empirical distribution of the sample could be used as an approximation to the distribution of the future values of $$\xi$$. If one has a prior model for $$\xi$$, one could obtain a posteriori distribution by a Bayesian update.

Discretization
To solve the two-stage stochastic problem numerically, one often needs to assume that the random vector $$\xi$$ has a finite number of possible realizations, called scenarios, say $$\xi_1,\dots,\xi_K$$, with respective probability masses $$p_1,\dots,p_K$$. Then the expectation in the first-stage problem's objective function can be written as the summation: $$ E[Q(x,\xi)]=\sum\limits_{k=1}^{K} p_kQ(x,\xi_k) $$ and, moreover, the two-stage problem can be formulated as one large linear programming problem (this is called the deterministic equivalent of the original problem, see section ).

When $$\xi$$ has an infinite (or very large) number of possible realizations the standard approach is then to represent this distribution by scenarios. This approach raises three questions, namely:

 * 1) How to construct scenarios, see ;
 * 2) How to solve the deterministic equivalent. Optimizers such as CPLEX, and GLPK can solve large linear/nonlinear problems. The NEOS Server, hosted at the University of Wisconsin, Madison, allows free access to many modern solvers. The structure of a deterministic equivalent is particularly amenable to apply decomposition methods, such as Benders' decomposition or scenario decomposition;
 * 3) How to measure quality of the obtained solution with respect to the "true" optimum.

These questions are not independent. For example, the number of scenarios constructed will affect both the tractability of the deterministic equivalent and the quality of the obtained solutions.

Stochastic linear programming
A stochastic linear program is a specific instance of the classical two-stage stochastic program. A stochastic LP is built from a collection of multi-period linear programs (LPs), each having the same structure but somewhat different data. The $$k^{th}$$ two-period LP, representing the $$k^{th}$$ scenario, may be regarded as having the following form:

$$	\begin{array}{lccccccc} \text{Minimize} & f^T x & + & g^T y & + & h_k^Tz_k & &  \\ \text{subject to} & Tx & + & Uy & &  & = & r \\ & &  & V_k y & + & W_kz_k & = & s_k \\ & x &, & y & , & z_k & \geq & 0 \end{array} $$

The vectors $$x$$ and $$y$$ contain the first-period variables, whose values must be chosen immediately. The vector $$z_k$$ contains all of the variables for subsequent periods. The constraints $$Tx + Uy = r$$ involve only first-period variables and are the same in every scenario. The other constraints involve variables of later periods and differ in some respects from scenario to scenario, reflecting uncertainty about the future.

Note that solving the $$k^{th}$$ two-period LP is equivalent to assuming the $$k^{th}$$ scenario in the second period with no uncertainty. In order to incorporate uncertainties in the second stage, one should assign probabilities to different scenarios and solve the corresponding deterministic equivalent.

Deterministic equivalent of a stochastic problem
With a finite number of scenarios, two-stage stochastic linear programs can be modelled as large linear programming problems. This formulation is often called the deterministic equivalent linear program, or abbreviated to deterministic equivalent. (Strictly speaking a deterministic equivalent is any mathematical program that can be used to compute the optimal first-stage decision, so these will exist for continuous probability distributions as well, when one can represent the second-stage cost in some closed form.) For example, to form the deterministic equivalent to the above stochastic linear program, we assign a probability $$p_k$$ to each scenario $$k=1,\dots,K$$. Then we can minimize the expected value of the objective, subject to the constraints from all scenarios:

$$ \begin{array}{lccccccccccccc} \text{Minimize} & f^\top x & + & g^\top y & + & p_1h_1^\top z_1 & + & p_2h_2^Tz_2 & + & \cdots & + & p_Kh_K^\top z_K & &  \\ \text{subject to} & Tx & + & Uy & &  &  &  &  &  &  &  & = & r \\ & &  & V_1 y & + & W_1z_1 &  &  &  &  &  &  & = & s_1 \\ & &  & V_2 y &  &  & + & W_2z_2 &  &  &  &  & = & s_2 \\ & &  & \vdots &  &  &  &  &  & \ddots &  &  &  & \vdots \\ & &  & V_Ky &  &  &  &  &  &  & + & W_Kz_K & = & s_K \\ & x &, & y & , & z_1 & , & z_2 & , & \ldots & , & z_K & \geq & 0 \\ \end{array} $$

We have a different vector $$z_k$$ of later-period variables for each scenario $$k$$. The first-period variables $$x$$ and $$y$$ are the same in every scenario, however, because we must make a decision for the first period before we know which scenario will be realized. As a result, the constraints involving just $$x$$ and $$y$$ need only be specified once, while the remaining constraints must be given separately for each scenario.

Scenario construction
In practice it might be possible to construct scenarios by eliciting experts' opinions on the future. The number of constructed scenarios should be relatively modest so that the obtained deterministic equivalent can be solved with reasonable computational effort. It is often claimed that a solution that is optimal using only a few scenarios provides more adaptable plans than one that assumes a single scenario only. In some cases such a claim could be verified by a simulation. In theory some measures of guarantee that an obtained solution solves the original problem with reasonable accuracy. Typically in applications only the first stage optimal solution $$x^*$$ has a practical value since almost always a "true" realization of the random data will be different from the set of constructed (generated) scenarios.

Suppose $$\xi$$ contains $$d$$ independent random components, each of which has three possible realizations (for example, future realizations of each random parameters are classified as low, medium and high), then the total number of scenarios is $$K=3^d$$. Such exponential growth of the number of scenarios makes model development using expert opinion very difficult even for reasonable size $$d$$. The situation becomes even worse if some random components of $$\xi$$ have continuous distributions.

Monte Carlo sampling and Sample Average Approximation (SAA) Method
A common approach to reduce the scenario set to a manageable size is by using Monte Carlo simulation. Suppose the total number of scenarios is very large or even infinite. Suppose further that we can generate a sample $$\xi^1,\xi^2,\dots,\xi^N$$ of $$N$$ replications of the random vector $$\xi$$. Usually the sample is assumed to be independent and identically distributed (i.i.d sample). Given a sample, the expectation function $$q(x)=E[Q(x,\xi)]$$ is approximated by the sample average

$$ \hat{q}_N(x) = \frac{1}{N} \sum_{j=1}^N Q(x,\xi^j) $$

and consequently the first-stage problem is given by

$$ \begin{array}{rlrrr} \hat{g}_N(x)=&\min\limits_{x\in \mathbb{R}^n}  & c^T x + \frac{1}{N} \sum_{j=1}^N Q(x,\xi^j)    &   \\ &\text{subject to} & Ax   &=&    b \\ &		   & x     &\geq& 0 \end{array} $$

This formulation is known as the Sample Average Approximation method. The SAA problem is a function of the considered sample and in that sense is random. For a given sample $$\xi^1,\xi^2,\dots,\xi^N$$ the SAA problem is of the same form as a two-stage stochastic linear programming problem with the scenarios $$\xi^j$$., $$j=1,\dots,N$$, each taken with the same probability $$p_j=\frac{1}{N}$$.

Statistical inference
Consider the following stochastic programming problem

$$ \min\limits_{x\in X}\{ g(x) = f(x)+E[Q(x,\xi)] \} $$

Here $$X$$ is a nonempty closed subset of $$\mathbb{R}^n$$, $$\xi$$ is a random vector whose probability distribution $$P$$ is supported on a set $$\Xi \subset \mathbb{R}^d$$, and $$Q: X \times \Xi \rightarrow \mathbb{R}$$. In the framework of two-stage stochastic programming, $$Q(x,\xi)$$ is given by the optimal value of the corresponding second-stage problem.

Assume that $$g(x)$$ is well defined and finite valued for all $$x\in X$$. This implies that for every $$x\in X$$ the value $$Q(x,\xi)$$ is finite almost surely.

Suppose that we have a sample $$\xi^1,\dots,\xi^N$$ of $$N$$realizations of the random vector $$\xi$$. This random sample can be viewed as historical data of $$N$$ observations of $$\xi$$, or it can be generated by Monte Carlo sampling techniques. Then we can formulate a corresponding sample average approximation

$$ \min\limits_{x\in X}\{ \hat{g}_N(x) = f(x)+\frac{1}{N} \sum_{j=1}^N Q(x,\xi^j) \} $$

By the Law of Large Numbers we have that, under some regularity conditions $$\frac{1}{N} \sum_{j=1}^N Q(x,\xi^j)$$ converges pointwise with probability 1 to $$E[Q(x,\xi)]$$ as $$N \rightarrow \infty$$. Moreover, under mild additional conditions the convergence is uniform. We also have $$E[\hat{g}_N(x)]=g(x)$$, i.e., $$\hat{g}_N(x)$$ is an unbiased estimator of $$g(x)$$. Therefore, it is natural to expect that the optimal value and optimal solutions of the SAA problem converge to their counterparts of the true problem as $$N \rightarrow \infty$$.

Consistency of SAA estimators
Suppose the feasible set $$X$$ of the SAA problem is fixed, i.e., it is independent of the sample. Let $$\vartheta^*$$ and $$S^*$$ be the optimal value and the set of optimal solutions, respectively, of the true problem and let $$\hat{\vartheta}_N$$ and $$\hat{S}_N$$ be the optimal value and the set of optimal solutions, respectively, of the SAA problem.

 * 1) Let $$g: X \rightarrow \mathbb{R}$$ and $$\hat{g}_N: X \rightarrow \mathbb{R}$$ be a sequence of (deterministic) real valued functions. The following two properties are equivalent:
 * 2) * for any $$\overline{x}\in X$$ and any sequence $$\{x_N\}\subset X$$ converging to $$\overline{x}$$ it follows that $$\hat{g}_N(x_N)$$ converges to $$g(\overline{x})$$
 * 3) * the function $$g(\cdot)$$ is continuous on $$X$$ and $$\hat{g}_N(\cdot)$$ converges to $$g(\cdot)$$ uniformly on any compact subset of $$X$$
 * 4) If the objective of the SAA problem $$\hat{g}_N(x)$$ converges to the true problem's objective $$g(x)$$ with probability 1, as $$N \rightarrow \infty$$, uniformly on the feasible set $$X$$. Then $$\hat{\vartheta}_N$$ converges to $$\vartheta^*$$ with probability 1 as $$N \rightarrow \infty$$.
 * 5) Suppose that there exists a compact set $$C \subset \mathbb{R}^n$$ such that
 * 6) * the set $$S$$ of optimal solutions of the true problem is nonempty and is contained in $$C$$
 * 7) * the function $$g(x)$$ is finite valued and continuous on $$C$$
 * 8) * the sequence of functions $$\hat{g}_N(x)$$ converges to $$g(x)$$ with probability 1, as $$N \rightarrow \infty$$, uniformly in $$x\in C$$
 * 9) * for $$N$$ large enough the set $$\hat{S}_N$$ is nonempty and $$\hat{S}_N \subset C$$ with probability 1
 * then $$\hat{\vartheta}_N \rightarrow \vartheta^*$$ and $$\mathbb{D}(S^*,\hat{S}_N)\rightarrow 0 $$ with probability 1 as $$N\rightarrow \infty $$. Note that $$\mathbb{D}(A,B) $$ denotes the deviation of set $$A$$ from set $$B$$, defined as

$$ \mathbb{D}(A,B) := \sup_{x\in A} \{ \inf_{x' \in B} \|x-x'\| \} $$

In some situations the feasible set $$X$$ of the SAA problem is estimated, then the corresponding SAA problem takes the form

$$ \min_{x\in X_N} \hat{g}_N(x) $$

where $$X_N$$ is a subset of $$\mathbb{R}^n$$ depending on the sample and therefore is random. Nevertheless, consistency results for SAA estimators can still be derived under some additional assumptions:
 * 1) Suppose that there exists a compact set $$C \subset \mathbb{R}^n$$ such that
 * 2) * the set $$S$$ of optimal solutions of the true problem is nonempty and is contained in $$C$$
 * 3) * the function $$g(x)$$ is finite valued and continuous on $$C$$
 * 4) * the sequence of functions $$\hat{g}_N(x)$$ converges to $$g(x)$$ with probability 1, as $$N \rightarrow \infty$$, uniformly in $$x\in C$$
 * 5) * for $$N$$ large enough the set $$\hat{S}_N$$ is nonempty and $$\hat{S}_N \subset C$$ with probability 1
 * 6) * if $$ x_N \in X_N$$ and $$ x_N $$ converges with probability 1 to a point $$ x$$, then $$ x \in X$$
 * 7) * for some point $$ x \in S^*$$ there exists a sequence $$ x_N \in X_N$$ such that $$ x_N \rightarrow x$$ with probability 1.
 * then $$\hat{\vartheta}_N \rightarrow \vartheta^*$$ and $$\mathbb{D}(S^*,\hat{S}_N)\rightarrow 0 $$ with probability 1 as $$N\rightarrow \infty $$.

Asymptotics of the SAA optimal value
Suppose the sample $$\xi^1,\dots,\xi^N$$ is i.i.d. and fix a point $$x \in X$$. Then the sample average estimator $$\hat{g}_N(x)$$, of $$g(x)$$, is unbiased and has variance $$\frac{1}{N}\sigma^2(x)$$, where $$\sigma^2(x):=Var[Q(x,\xi)]$$ is supposed to be finite. Moreover, by the central limit theorem we have that

$$ \sqrt{N} [\hat{g}_N- g(x)] \xrightarrow{\mathcal{D}} Y_x $$

where $$\xrightarrow{\mathcal{D}}$$ denotes convergence in distribution and $$Y_x$$ has a normal distribution with mean $$0$$ and variance $$\sigma^2(x)$$, written as $$\mathcal{N}(0,\sigma^2(x))$$.

In other words, $$\hat{g}_N(x)$$ has asymptotically normal distribution, i.e., for large $$N$$, $$\hat{g}_N(x)$$ has approximately normal distribution with mean $$g(x)$$ and variance $$\frac{1}{N}\sigma^2(x)$$. This leads to the following (approximate) $$100(1-\alpha)$$% confidence interval for $$f(x)$$:

$$ \left[ \hat{g}_N(x)-z_{\alpha/2} \frac{\hat{\sigma}(x)}{\sqrt{N}}, \hat{g}_N(x)+z_{\alpha/2} \frac{\hat{\sigma}(x)}{\sqrt{N}}\right] $$

where $$z_{\alpha/2}:=\Phi^{-1}(1-\alpha/2)$$ (here $$\Phi(\cdot)$$ denotes the cdf of the standard normal distribution) and

$$ \hat{\sigma}^2(x) := \frac{1}{N-1}\sum_{j=1}^{N} \left[ Q(x,\xi^j)-\frac{1}{N} \sum_{j=1}^N Q(x,\xi^j) \right]^2 $$

is the sample variance estimate of $$\sigma^2(x)$$. That is, the error of estimation of $$g(x)$$ is (stochastically) of order $$ O(\sqrt{N})$$.

Biological applications
Stochastic dynamic programming is frequently used to model animal behaviour in such fields as behavioural ecology. Empirical tests of models of optimal foraging, life-history transitions such as fledging in birds and egg laying in parasitoid wasps have shown the value of this modelling technique in explaining the evolution of behavioural decision making. These models are typically many-staged, rather than two-staged.

Economic applications
Stochastic dynamic programming is a useful tool in understanding decision making under uncertainty. The accumulation of capital stock under uncertainty is one example; often it is used by resource economists to analyze bioeconomic problems where the uncertainty enters in such as weather, etc.

Example: multistage portfolio optimization
The following is an example from finance of multi-stage stochastic programming. Suppose that at time $$t=0$$ we have initial capital $$W_0$$ to invest in $$n$$ assets. Suppose further that we are allowed to rebalance our portfolio at times $$t=1,\dots,T-1$$ but without injecting additional cash into it. At each period $$t$$ we make a decision about redistributing the current wealth $$W_t$$ among the $$n$$ assets. Let $$x_0=(x_{10},\dots,x_{n0})$$ be the initial amounts invested in the n assets. We require that each $$x_{i0}$$ is nonnegative and that the balance equation $$\sum_{i=1}^{n}x_{i0}=W_0$$ should hold.

Consider the total returns $$\xi_t=(\xi_{1t},\dots,\xi_{nt})$$ for each period $$t=1,\dots,T$$. This forms a vector-valued random process $$\xi_1,\dots,\xi_T$$. At time period $$t=1$$, we can rebalance the portfolio by specifying the amounts $$x_1=(x_{11},\dots,x_{n1})$$ invested in the respective assets. At that time the returns in the first period have been realized so it is reasonable to use this information in the rebalancing decision. Thus, the second-stage decisions, at time $$t=1$$, are actually functions of realization of the random vector $$\xi_1$$, i.e., $$x_1=x_1(\xi_1)$$. Similarly, at time $$t$$ the decision $$x_t=(x_{1t},\dots,x_{nt})$$ is a function $$x_t=x_t(\xi_{[t]})$$ of the available information given by $$\xi_{[t]}=(\xi_{1},\dots,\xi_{t})$$ the history of the random process up to time $$t$$. A sequence of functions $$x_t=x_t(\xi_{[t]})$$, $$t=0,\dots,T-1$$, with $$x_0$$ being constant, defines an implementable policy of the decision process. It is said that such a policy is feasible if it satisfies the model constraints with probability 1, i.e., the nonnegativity constraints $$x_{it}(\xi_{[t]})\geq 0$$, $$i=1,\dots,n$$, $$t=0,\dots,T-1$$, and the balance of wealth constraints,

\sum_{i=1}^{n}x_{it}(\xi_{[t]}) = W_t, $$

where in period $$t=1,\dots,T$$ the wealth $$W_t$$ is given by

W_t = \sum_{i=1}^{n}\xi_{it} x_{i,t-1}(\xi_{[t-1]}), $$

which depends on the realization of the random process and the decisions up to time $$t$$.

Suppose the objective is to maximize the expected utility of this wealth at the last period, that is, to consider the problem

\max E[U(W_T)]. $$

This is a multistage stochastic programming problem, where stages are numbered from $$t=0$$ to $$t=T-1$$. Optimization is performed over all implementable and feasible policies. To complete the problem description one also needs to define the probability distribution of the random process $$\xi_1,\dots,\xi_T$$. This can be done in various ways. For example, one can construct a particular scenario tree defining time evolution of the process. If at every stage the random return of each asset is allowed to have two continuations, independent of other assets, then the total number of scenarios is $$2^{nT}.$$

In order to write dynamic programming equations, consider the above multistage problem backward in time. At the last stage $$t=T-1$$, a realization $$\xi_{[T-1]}=(\xi_{1},\dots,\xi_{T-1})$$ of the random process is known and $$x_{T-2}$$ has been chosen. Therefore, one needs to solve the following problem

\begin{array}{lrclr} \max\limits_{x_{T-1}}  & E[U(W_T)|\xi_{[T-1]}]    &   \\ \text{subject to} & W_T  &=&    \sum_{i=1}^{n}\xi_{iT}x_{i,T-1} \\ &\sum_{i=1}^{n}x_{i,T-1}&=&W_{T-1}\\ & x_{T-1}    &\geq& 0 \end{array} $$

where $$E[U(W_T)|\xi_{[T-1]}]$$ denotes the conditional expectation of $$U(W_T)$$ given $$\xi_{[T-1]}$$. The optimal value of the above problem depends on $$W_{T-1}$$ and $$\xi_{[T-1]}$$ and is denoted $$Q_{T-1}(W_{T-1},\xi_{[T-1]})$$.

Similarly, at stages $$t=T-2,\dots,1$$, one should solve the problem

\begin{array}{lrclr} \max\limits_{x_{t}}  & E[Q_{t+1}(W_{t+1},\xi_{[t+1]})|\xi_{[t]}]    &   \\ \text{subject to} & W_{t+1}  &=&    \sum_{i=1}^{n}\xi_{i,t+1}x_{i,t} \\ &\sum_{i=1}^{n}x_{i,t}&=&W_{t}\\ & x_{t}    &\geq& 0 \end{array} $$

whose optimal value is denoted by $$Q_{t}(W_{t},\xi_{[t]})$$. Finally, at stage $$t=0$$, one solves the problem

\begin{array}{lrclr} \max\limits_{x_{0}}  & E[Q_{1}(W_{1},\xi_{[1]})]    &   \\ \text{subject to} & W_{1}  &=&    \sum_{i=1}^{n}\xi_{i,1}x_{i0} \\ &\sum_{i=1}^{n}x_{i0}&=&W_{0}\\ & x_{0}    &\geq& 0 \end{array} $$

Stagewise independent random process
For a general distribution of the process $$\xi_t$$, it may be hard to solve these dynamic programming equations. The situation simplifies dramatically if the process $$\xi_t$$ is stagewise independent, i.e., $$\xi_t$$ is (stochastically) independent of $$\xi_1,\dots,\xi_{t-1}$$ for $$t=2,\dots,T$$. In this case, the corresponding conditional expectations become unconditional expectations, and the function $$Q_t(W_t)$$, $$t=1,\dots,T-1$$ does not depend on $$\xi_{[t]}$$. That is, $$Q_{T-1}(W_{T-1})$$ is the optimal value of the problem

\begin{array}{lrclr} \max\limits_{x_{T-1}}  & E[U(W_T)]    &   \\ \text{subject to} & W_T  &=&    \sum_{i=1}^{n}\xi_{iT}x_{i,T-1} \\ &\sum_{i=1}^{n}x_{i,T-1}&=&W_{T-1}\\ & x_{T-1}    &\geq& 0 \end{array} $$

and $$Q_t(W_t)$$ is the optimal value of

\begin{array}{lrclr} \max\limits_{x_{t}}  & E[Q_{t+1}(W_{t+1})]    &   \\ \text{subject to} & W_{t+1}  &=&    \sum_{i=1}^{n}\xi_{i,t+1}x_{i,t} \\ &\sum_{i=1}^{n}x_{i,t}&=&W_{t}\\ & x_{t}    &\geq& 0 \end{array} $$

for $$t=T-2,\dots,1$$.

Modelling languages
All discrete stochastic programming problems can be represented with any algebraic modeling language, manually implementing explicit or implicit non-anticipativity to make sure the resulting model respects the structure of the information made available at each stage. An instance of an SP problem generated by a general modelling language tends to grow quite large (linearly in the number of scenarios), and its matrix loses the structure that is intrinsic to this class of problems, which could otherwise be exploited at solution time by specific decomposition algorithms. Extensions to modelling languages specifically designed for SP are starting to appear, see: They both can generate SMPS instance level format, which conveys in a non-redundant form the structure of the problem to the solver.
 * AIMMS - supports the definition of SP problems
 * EMP SP (Extended Mathematical Programming for Stochastic Programming) - a module of GAMS created to facilitate stochastic programming (includes keywords for parametric distributions, chance constraints and risk measures such as Value at risk and  Expected shortfall).
 * SAMPL - a set of extensions to AMPL specifically designed to express stochastic programs (includes syntax for chance constraints, integrated chance constraints and Robust Optimization problems)

title: Stochastic programming


text:
Lanchester submachine gun

The Lanchester is a submachine gun ("machine carbine") manufactured by the Sterling Armaments Company between 1941 and 1945. It is an evolution from MP28/II and was manufactured in two versions, Mk.1 and Mk.1*; the latter was a simplified version of the original Mk.1, with no fire selector and simplified sights. It was primarily used by the Royal Navy during the Second World War, and to a lesser extent by the Royal Air Force Regiment (for airfield protection). It was given the general designation of Lanchester after George Lanchester, who was charged with producing the weapon at the Sterling Armaments Company.

History
Following the Dunkirk evacuation in 1940, the Royal Air Force decided it required a submachine gun for airfield defence. With no time available for the usual research and development of a new weapon, it was decided to create a gun based on the German MP 28, after looking at two examples from Ethiopia. The British Admiralty decided to join with the RAF in adopting the new weapon, and played a key role in its design. Ultimately, it was within the Royal Navy that most of the Lanchesters that were produced went into service.

It was given the general designation of Lanchester after George Herbert Lanchester, who was charged with producing the weapon at the Sterling Armaments Company, the same company that later produced the Sterling submachine gun.

The Lanchester was envisioned as a weapon used for guarding prisoners and accompanying naval landing and assault parties. It was a very solid, well-made submachine gun of high-quality materials, in many ways the complete opposite of its direct contemporary, the Sten.

The Lanchester had a heavy wooden butt and stock, which was for simplicity copied from the British SMLE service rifle, a machined-steel action and breech block, a magazine housing made from solid brass (later steel) and a mounting on the muzzle for use of a long-bladed 1907 bayonet. The rifling differed from the German original in details to accommodate various lots of 9mm ammunition then being acquired for service use.

Produced in two versions, Mk.1 and Mk.1*. The Mk.1* was a simplified version of the original Mk.1, which omitted the fire mode selector (full automatic only) and used simplified sights.

Production

The first contract of 13 June 1941 produced an initial 50,000 Lanchesters that were nearly all for the Royal Navy use. The British Army by now had supplies of the relatively expensive US-produced Thompson SMG, whereas the RAF had chosen to instead test the S&W Model 1940 Light Rifle (also US-produced, chambered in 9mm), whose use was not adopted and was mostly discarded.

The final contract was issued on 9 October 1943. Production averaged 3,410 units per month over 28 months. According to contract records, Sterling was to have made guns serially numbered from 1 to 9999, then (S) A1 to about A64580. Certain numbers of Mk.1 were modified later in the war and designated Mk.1*, the key differences being the removal of the fire-selector switch and addition of simplified rear sights. This modification makes it difficult to ascertain exactly the production runs for each model.

There were four Lanchester assembly plants, though Lanchester assembly contracts were actually awarded to only three firms. Sterling assembly of the Lanchester was split between the Sterling Engineering Company Ltd in Dagenham (code S109), and the Sterling Armaments Company in Northampton (code M619).

Quantities produced:
 * Sterling (two factories; codes S109 and M619): approx. 74,579.
 * Greener (code M94): approx. 16,990.
 * Boss (code S156): approx. 3,900.

Some early version do not appear to be code marked at all except by the serial number prefix of "S", "A", or "SA". Pieces marked with the "A" prefix lacked perfect interchangeability and often required hand fitting of components.

Markings
The year of manufacture of any particular Lanchester can be found stamped in small almost indistinguishable numbers next to the crossed flags military proof mark on the top of the rearmost magazine housing flange.

Sterling-made MK.1 Lanchester guns are marked on top of the magazine housing as follows:
 * LANCHESTER
 * MK.I
 * SA.
 * XXXX

("S" indicates Sterling manufacture and "A" indicates serial number prefix)

Operation
The Lanchester is an open-bolt, self-loading blowback-operated weapon with a selective-fire option (located in front of the trigger) on early versions. A tubular receiver was attached to the front of the wooden stock, which could be pivoted barrel down for maintenance and disassembly. The wooden stock was patterned after that of the Lee–Enfield rifle, and a bayonet lug centred below the muzzle accepted the Pattern 1907 sword-bayonet as used on the Lee–Enfield No. 1 Mk. III* (previously called the S.M.L.E.)

It used a straight 50-round magazine containing 9×19mm Parabellum cartridges (special pouches were produced to hold three magazines each) which fit into the magazine housing from the left, with spent cartridges ejected on the right. It was interchangeable with the shorter 32-round Sten magazine. A magazine loading tool was needed to load both 32- and 50-round magazines more easily. One of the two magazine pouches had a special pocket on the front for this loader.

Mk.1s featured a front blade sight with adjustable rifle-type sights, marked between 100 and 600 yards. Mk.1* featured a much simplified flip-up sight marked 100 or 200 yards.

Manual safety is made in the form of locking cut, made in the receiver, which engages the bolt handle to lock bolt in open (cocked) position. It proved notoriously susceptible to accidental discharge if the weapon were dropped. For cleaning, the weapon had a brass oiler bottle and pull through held inside the butt stock (similar to the Lee–Enfield rifle).

Service
The Lanchester was in service with the Royal Navy, Royal Canadian Navy and other Commonwealth navies throughout the war and for some decades after. The last examples left Royal Naval service in the 1970s and are now collectors' items.

A large number of Lanchesters were subsequently sold off to foreign nations. These are often marked with two broad arrows, point-to-point (appearing as a six-pointed star), stamped just before the serial number. This symbol is sometimes accompanied by the letter "S" and denotes "Sold out of Service".

A number were acquired by Chile, Egypt and Argentina after purchasing former Royal Navy warships; those were often sold as “complete package” with everything aboard including the weapons locker.

During the 2023 Israel-Hamas war, a Lanchester was spotted in a weapons cache seized from slain Hamas fighters by the Israel Defense Forces. It was used during the Be'eri massacre as well.

Users

 * 🇦🇷 Argentina
 * 🇦🇺 Australia
 * Biafra
 * 🇨🇱 Chile:In service until the 1970s
 * 🇨🇦 Canada:In service with the Royal Canadian Navy until the 1960s
 * 🇨🇾 Cyprus
 * 🇩🇴 Dominican Republic: Bought by the Dominican Army in the 1950s; used by the Constitutionalists in the Dominican Civil War of 1965.
 * 🇪🇬 Egypt:Inherited from British forces after independence; more were purchased between 1946 and 1947. Used by frontline military units until 1967. In 1969 remaining Lanchesters were transferred to the Central Security Forces; they remained in service with police until the 1980s.
 * 🇬🇷 Greece Hellenic Navy
 * 🇮🇩 Indonesia (captured from Dutch)
 * 🇲🇲 Myanmar:Retired.
 * 🇳🇱 Netherlands
 * Dutch East Indies
 * 🇳🇿 New Zealand
 * 🇬🇧 United Kingdom
 * Hamas

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

title: Lanchester submachine gun


text:
The Operational Art of War

The Operational Art of War (TOAW) is a series of computer wargames noted for their scope, detail, and flexibility in recreating, at an operational level, the major land battles of the 20th century. A Norm Koger design, TalonSoft published the first of the series in 1998. Matrix Games bought the rights to the franchise and created a new game in 2006, TOAW 3, which was the first non-Norm Koger designed game in the series.

Games in the series
TalonSoft published: Matrix Games published:
 * The Operational Art of War Vol. 1: 1939–1955 (1998)
 * The Operational Art of War II: Modern Battles 1956–2000 (1999)
 * The Operational Art of War II: Flashpoint Kosovo (1999)
 * The Operational Art of War II: Elite Edition (2000)
 * The Operational Art of War: Century of Warfare (2000)
 * The Operational Art of War III  (2006)
 * The Operational Art of War IV (2017)

Concept
The basic appearance of the game is the traditional view onto a hexagonal grid, although the player may choose a map-like overhead view with military symbols and basic info for the units, or an isometric view that depicts the units with small pictures of soldiers, tanks, etc. Gameplay is turn-based.

The scale of the game is variable, with distances ranging from 2.5 km per hex to 50 km per hex, and each turn simulating from 1/4 day to 1 week of time, but is fundamentally "operational", focusing on battalion, division, and corps combat. The option of scale is left to a maker of a particular scenario to choose, resulting in a wide range of user-made scenarios; ranging from, for example, a small engagement in northern Germany between several companies to an entire World War II on division scale.

The maximum number of units that can be made in a scenario was 2,000 per side until TOAW IV, although managing more than 200 can often be complicated. Each unit is assigned unique equipment (types of infantry, tanks, aircraft, etc.) and given its own name, info and color code.

The game also includes "events", which is a series of programmable events which display a message and can have several different causes and effects. The variability of these events makes each scenario—when properly designed—very complex and variable. The maximum number of in-game events is 500 (or 1,000 for TOAW III version).

The games include a scenario editor, and much of the content in the follow-up games are designs developed by the community of avid players.

Version IV
Version IV was released November 2017, and included a large number of changes, among which are:
 * unit limit increased from 2,000 to 10,000
 * event limit increased from 999 to 10,000
 * significant changes to naval combat
 * significant changes to how combat uses turn time
 * supply system

Volume 1
In the United States, The Operational Art of War sold 12,789 copies during 1998. These sales accounted for $555,681 in revenue that year.

Volume 2
In the United States, The Operational Art of War Volume II sold 1,298 copies during 1999.

title: The Operational Art of War


text:
Talk:Norman Tebbit

Untitled
What is the evidence to support the assertion that Tebbitt was a pilot ?. It is my understanding that he was cabin crew

In his autobiography he says he was a pilot with the RAF in his national service and then with BOAC.

Cabin crew?? Really?? Bloody hell...imagine Norman Tebbit mincing down the ailses offering you a British Airways "all day Deli" bag!

''He was then a journalist for the Financial Times before serving with the RAF, flying Meteor and Vampire jets during four years of National Service. On leaving the RAF he joined BOAC in 1953 as a pilot'' And all by the age of 22, if the dates here are right. Are the dates right?

 * See edit. He joined BOAC in 1953 but continued in the Auxiliary Air Force.
 * Gravuritas (talk) 00:42, 25 August 2013 (UTC)

Quotes
Should the quotes section be moved to Wikiquote?--Darrelljon 20:59, 26 June 2006 (UTC)

 * Agree, usual WP practice. Philip Cross 21:47, 26 June 2006 (UTC)

This quote used to be around but it's now not even on wikiquote. I can't find a decent provenance for the (complete including last bit) quote on the net.

"The Muslim religion is so unreformed since it was created that nowhere in the Muslim world has there been any real advance in science, or art or literature, or technology in the last 500 years...we've leapt ahead in all material terms, but the Muslim world would say we have fallen down in all spiritual and moral terms. We have to accept our share of the blame and they have to accept theirs."

Is it apocryphal? Cutler (talk) 00:02, 27 March 2008 (UTC)
 * He said it during an interview with the ePolitix website in August 2005.--Johnbull (talk) 12:58, 27 March 2008 (UTC)

transgenderphobe?
Is he a transgender phobe when he refered to gender reasingment surgery as a form of mutilation when trying to attach his wrecking amendment to the Gender Recognition Act.--Lucy-marie 20:26, 4 September 2006 (UTC)
 * Tebbit seems to be phobic towards any number of people who fall short of his definition of 'normal', and it's often something to do with sexuality/gender that bothers him. He once praised Ian Duncan-Smith for being 'remarkably normal', on the grounds that he is married with children, while criticising Michael Portillo, the implication being obvious. Some of Tebbit's statements over the years are truly extraordinary. MarkB79 03:06, 2 November 2006 (UTC)

Trade unionist?
This conversation recently occured at Talk:Margaret Thatcher as well - should Tebbit be included in the Category:British trade unionists? I would think he could perhaps be more accurately described as Category:British anti-trade unionists? :) --Bookandcoffee 20:04, 10 October 2006 (UTC)
 * There is no way Tebbit should be in this category, it's absurd. Individuals should not qualify for the list of British Trade Unionists just because they happened to be members of a trade union at some point in their lives, you'd have to include thousands of people on here - it should be restricted to people who are particularly notable for being trade unionists and/or have played an important role within the union movement. Listing Tebbit as a notable 'trade unionist' is ridiculous.

MarkB79 03:06, 2 November 2006 (UTC)

I agree, see further down at Talk:Norman Tebbit -84user (talk) 18:40, 9 July 2008 (UTC)

Very Militant Trade Unionist
He was a prominent leader of BALPA in the early 1960's when the union refused to co-operate in introducing the new big jet aircraft. He spent months leading this 'go slow/work to rule' against the government and the people when British Airways was publicly owned. 82.47.176.254 21:05, 14 November 2006 (UTC)mikeL
 * Cite. See further down at Talk:Norman Tebbit -84user (talk) 18:40, 9 July 2008 (UTC)

POV Inherent in choice of quotes
The selection of quotes from Tebbit cited in the last section of this article has clearly been carefully picked (probably out of context) to make him look bad. This is a very sneaky way of inserting a leftwing POV into the article, as the quotes themselves can't be challenged. It is urgently necessary to add some more quotes that present a more accurate and well-rounded view of Tebbit's perspective. Walton monarchist89 10:41, 17 January 2007 (UTC)

 * Personally, I think they are excellent quotes. His comments on the BBC are ones that any serious Right-Winger will agree with. His comments on The West vs. Islam are also right on the money. Only a pathetic pinko could view those quotes as anything other than the truth. 87.194.74.193 01:19, 10 February 2007 (UTC)

 * The comments you have just made about other users points of views may constiue a personal attack and should be avoided.--Lucy-marie 14:22, 10 February 2007 (UTC)

 * In fact, I resent the allegation that I am a "pathetic pinko" - I am in fact a passionate Conservative and Thatcherite. Although I myself agree with many of the opinions expressed in his quotes, I'm only trying to say that they don't represent a balanced view of his opinions - he has said many more moderate things, that the article glosses over. I fear you've misunderstood my comments; I wasn't trying to attack Tebbit, but to criticise the way this article is organised. Walton monarchist89 10:48, 22 February 2007 (UTC)

Removed quotes section entirely. Seems like the simplest way when Wikiquote exists. If someone wants to add something more balanced, feel free 82.152.90.185 09:57, 22 August 2007 (UTC)

Atheist?
This article has been placed in the category of British atheists. Nowhere in the text can I find this mentioned, let alone cited. If there were an unsourced comment in the body of the article to the effect that Tebbit is an atheist then I would mark it with. I have heard that he is not religious and so it's certainly possible that he is an atheist. However, without a citation, I submit that the article should be removed from the British atheists category. Greenshed 20:16, 1 July 2007 (UTC)

When he was interviewed by John Mortimer (as mentioned in the article) he said he wasn't an atheist and believed in a system of order in the universe, but he couldn't accept Christian doctrine (I can't remember the details of what he disagreed with there).

He further said, half-jokingly, that in his opinion God couldn't be a socialist because a socialist wouldn't have got rid of the dinosaurs or let them die out.

Meltingpot (talk) 20:49, 13 October 2009 (UTC)
 * Unless we have a citation from him saying he is an athiest then we shouldn't have him in the cat, perhaps look through the history and see who added it and ask thhem, or have a quick search and if you find nothing then remove it. Off2riorob (talk) 21:08, 13 October 2009 (UTC)

Category:British trade unionists?
I have removed this article from Category:British trade unionists because I could not see any verifiable cite that he was active as one. I have just added a proposal on category inclusion criteria to Category talk:British trade unionists. If the article gets any verifiable cite on notable union member activity, please reinsert. -84user (talk) 18:40, 9 July 2008 (UTC)
 * "The incompetence of management in BOAC steadily pushed me into a more active role in BALPA"(Tebbit, 1988, p58)
 * "....my BALPA activities... I became increasingly involved in the technical committee as well as the 707 pilots' council."(p60)
 * So the active trade unionist is demonstrated. Whether this, in the absence of his later political career, would be notable enough of itself to merit inclusion in the trade union list may be debatable.  However, I'd suggest that in the light of the 1982 employment act, then any trade union activism is indeed 'notable'.
 * Gravuritas (talk) 01:10, 25 August 2013 (UTC)

Category removal
I have also removed this article from these categories due to lack of verifiable cites: Hopefully I did this in accordance with these Wikipedia policies: -84user (talk) 19:27, 9 July 2008 (UTC)
 * Category:Daily Mail journalists
 * Category:British atheists
 * Verifiability
 * Notability (people)
 * WP:BLP
 * Categorization
 * Archived Wikipedia talk:Categorization

The cookery book
Anyone "game" to make reference to it? —Preceding unsigned comment added by 83.104.132.41 (talk) 12:53, 30 July 2009 (UTC)

National Service
Some claims in the article as it stood before don't quite add up. At that time national service in Britain only lasted 2 years, not 4, and it would have been unlikely although not impossible that a conscript flew as a pilot on advanced jets. Was he a volunteer? On the other hand, is it likely that a 16-year-old was a journalist on the Financial Times? Did he manage to get his conscription deferred for 2 years, and only served for 2 years? PatGallacher (talk) 13:19, 3 May 2012 (UTC)
 * Covered in Norman Tebbit Upwardly mobile. Joined the FT at 16, initially worked in prices dept on shares etc, composed an occasional paragraph. Bullied by NATSOPA into joining the union, swore he'd break the power of the closed shop.  Yes he was a volunteer in the RAF, joined 1949, 6 months Prentices, 6 months Harvards (total 18 months, somehow) then Meteors and so on.
 * Gravuritas (talk) 01:23, 25 August 2013 (UTC)

Butchery
The article has been butchered by a leftist. Section "Member of PArliament" has no relevant info, and is a bunch of POV. Like all leftists rants, the original bits are rubbish, and the readable bits are not original (to paraphrase Samuel Johnson).220.244.79.117 (talk) 03:54, 6 November 2012 (UTC)

Steel's comment on balance of trade deleted
Garbage comment deleted. See http://www.cbi.org.uk/media/1226070/cbi-rebalancing-chartbook.pdf see fig 3.2. There is only one year for which a major long-term trend develops that is statistically significant between 1970 and 2010, and that's 1997. Gravuritas (talk) 08:50, 13 October 2013 (UTC)

Request for Comments
There is an RfC on the question of using "Religion: None" vs. "Religion: None (atheist)" in the infobox on this and other similar pages.

The RfC is at Template talk:Infobox person.

Please help us determine consensus on this issue. --Guy Macon (talk) 22:05, 25 April 2015 (UTC)

External links modified
Hello fellow Wikipedians,

I have just added archive links to 2 one external links on Norman Tebbit. Please take a moment to review my edit. If necessary, add after the link to keep me from modifying it. Alternatively, you can add to keep me off the page altogether. I made the following changes:
 * Added archive https://web.archive.org/20111123122906/http://www.newstatesman.com/200006050013 to http://www.newstatesman.com/200006050013
 * Added archive https://web.archive.org/20090213222839/http://www.cpgb.org.uk:80/worker/386/mi6-ukip.html to http://www.cpgb.org.uk/worker/386/mi6-ukip.html

When you have finished reviewing my changes, please set the checked parameter below to true to let others know.

Cheers. —cyberbot II  Talk to my owner :Online 19:04, 28 August 2015 (UTC)

Memoirs?
You mention his memoirs. Are these still in print, or accessible online, as referenced in other politicians' wiki pages? Valetude (talk) 18:15, 12 February 2020 (UTC)

 * There seems to be a problem with the citations in this article. Although, as you say, his memoirs are mentioned, the citations all just say "Tebbit" with page number. But here is no year to identify which book it is. Eight books are listed. I would guess it's meant to be Upwardly Mobile (Futura, 1991) ISBN 0-297-79427-2, or Unfinished Business (Weidenfeld and Nicolson, 1991) ISBN 0-297-81149-5. 86.190.2.179 (talk) 18:32, 12 February 2020 (UTC)

No one with a conscience
This article should point out that Norman Tebbit said "Let's face it - no one with a conscience ever voted Tory" and it has been said that for once, this was a time when a Tory spoke the truth. Vorbee (talk) 06:38, 8 May 2020 (UTC)
 * It seems Hugo Young just made it up to please the lefty readers of ''The Grauniad"? Martinevans123 (talk) 07:58, 8 May 2020 (UTC)

Dead
He is reported to have died as of 24 June 2020. — Preceding unsigned comment added by 86.131.7.127 (talk) 13:50, 24 June 2020 (UTC)

Update re atheism and relationship with Dean at St Edmondsbury Cathedral
As a family member I have been asked to update the article on homosexuality and possibly atheism. Norman and Joe Hawes (the Dean at St Edmondsbury Cathedral) have reconciled their differences and Norman is now both an attendee at and benefactor to the Cathedral. Joe led the funeral service in January 2021 for Lady Tebbit and remarked to me then that "Norman has buried the hatchet - fortunately not between my shoulder blades as with political opponents". Since then there has been a friendship and to quote from the Daily Mail article of 30th January 2021 "....a few years ago the Dean at the Cathedral got to me" re God and religion.

What evidence would be needed to verify this - emails from both Norman and Joe?

Apologies if I'm not following convention here. I've never edited Wikipedia before and living person pages are understandably more sensitive. J2T2 (talk) 16:07, 14 April 2024 (UTC)

title: Talk:Norman Tebbit


text:
Berneray (North Uist)

Berneray (Beàrnaraigh na Hearadh) is an island and community in the Sound of Harris, Scotland. It is one of fifteen inhabited islands in the Outer Hebrides. It is famed for its rich and colourful history which has attracted much tourism. It lies within the South Lewis, Harris and North Uist National Scenic Area, one of 40 such areas in Scotland which are defined so as to identify areas of exceptional scenery and to ensure its protection from inappropriate development.

With an area of 10.1 km2, Berneray rises to a height of 305 ft at Beinn Shlèibhe (Moor Hill) and 278 ft at Cnoc Bhuirgh (Borve Hill). It is known for its sandy beaches backed with sand dunes. Tràigh Iar (West Beach), a three-mile stretch of wide, clean and often deserted sand, is widely acclaimed.

There is strong evidence that points to Berneray being inhabited since the Bronze Age, and possibly before. The island is scattered with ancient sacred sites, stone circles, signs of Viking inhabitation and historical buildings, some several centuries old.

Although Berneray is geographically closer to North Uist, it is part of the Bays of Harris Estate and under the Parish of Harris.

Etymology
The name "Berneray" is from the Old Norse Bjarnar-øy and means either "Bjorn's island" or possibly "bear island". This island is known by Gaelic speakers as Beàrnaraigh na Hearadh, meaning "Berneray of Harris" to distinguish it from Beàrnaraigh Cheann Bharraigh, literally "Berneray of Barra Head ", now usually known in English simply as Barra Head and Beàrnaraigh Leòdhais, know as Great Bernera. More recently Gaelic speakers have also used Beàrnaraigh Uibhist, i.e. "Berneray of Uist" to refer to the northern Berneray.

Demographics
In common with most islands in the Outer Hebrides, the population declined during the 19th and 20th centuries. However, the past few years has seen a stabilisation. The island's population was 138 as recorded by the 2011 census a small rise since 2001 when there were 136 usual residents. During the same period Scottish island populations as a whole grew by 4% to 103,702. Most people on Berneray speak Scottish Gaelic, many as a first language.

Commerce and transport
The main industries are fishing, crofting (small-scale individual farming), media/IT and tourism. Broadband internet provision became available in January 2006, giving an incentive to people wishing to relocate to Berneray and helping sustain the population and community.

A key feature of Berneray is its machair. The machair is a coastal plain made up of windblown shell sand. Traditional crofting practice, which involves summer agriculture using seaweed together with dung from winter grazing animals as natural fertiliser, has, over time, bound together and stabilised the land. The machair is ploughed in rotation, giving a patchwork of crops and fallow of different ages which supports a wide range of flowers. Berneray has a particularly fine machair, a result of careful husbandry by the island's crofters, helped by the absence of rabbits.

The youth hostel on Berneray is part of the Gatliff Hebridean Hostels Trust. The hostel consists of two restored black houses and is located at a magnificent setting overlooking the sound of Harris.

Possibly the greatest change in modern times occurred in 1999 when the causeway opened between Berneray and Otternish on North Uist. This has eased travelling on and off the island, improving employment prospects and accelerating the carriage of produce (notably, crabs and lobster). The causeway contains culverts that allow the easy passage of otters and fish from one side of the structure to the other. The causeway was formally opened by Prince Charles in April 1999. It is the first of a series of causeways connecting islands in the area. The North Ford Causeway (the A865) connects North Uist to the Isle of Benbecula. The South Ford Causeway (still the A865) connects Benbecula to South Uist. Finally, the Eriskay Causeway connects South Uist to Eriskay.

Berneray is served by regular local bus services from Lochmaddy on North Uist, many of which form part of the "Spine Route" from Eriskay. Most services are operated by Grenitote Travel, with funding from Comhairle nan Eilean Siar. Berneray is also linked to Harris, Scotland via Leverburgh by the ferry MV Loch Portain, operated by Caledonian MacBrayne.

The CalMac ferry MV Loch Bhrusda is named after the largest loch on Berneray. It can occasionally be seen relieving for MV Loch Portain.

Nature

The crofting practices also encourage a wide array of wildlife on Berneray. On early summer evenings you can sometimes hear snipe drumming, and even the rasp of a corncrake. Mute swans can be seen on Loch Brusda, and greylag geese are common. In the winter they are joined by barnacle, and a few brent geese. Ravens, grey herons and buzzards are often to be seen. Golden eagles and hen harriers are rarer sights, usually in the winter. Wading birds on the shore include redshank, sanderling, turnstone, oystercatcher, dunlin, curlew, whimbrel, ringed plover.

Further out, around the shores of Berneray, are mallards, eiders, red-breasted mergansers, and, more rarely, black-throated and great northern divers. Shags and cormorants fish in the seas around Berneray throughout the year, and in summer you can see gannets diving. Common seals often congregate at low tide on the rocks in Bays Loch, and can often be seen from the parking area a little way beyond the Post Office or by taking a boat trip out into the bay. Grey seals, which are larger and can be distinguished by the long "Roman" noses, also haul out there occasionally, but are more common off the West Beach. Though the otters of Berneray are out during the day more often than on the mainland, they are still elusive, and it takes patience and luck to see one.

Media and the arts
A documentary entitled Shepherds of Berneray was aired on UK television in 1981.

Berneray was in world news in 1987 when it was found that Charles, Prince of Wales had visited the island to live a normal Berneray life as a crofter. He lived and worked with a crofter for one week and his visit spawned the later television documentary, A Prince Among Islands, in 1991.

In 2007 a DVD called The Old New Year – a living tradition on the Isle of Berneray - was released about the island's Oidhche Challainn ("New Year's Eve", literally "Night of the Calends) celebration of New Year on 12 January in line the former Highland tradition of following the Julian Calendar.

In 2009 it was reported that Berneray's west beach had been used to advertise Kae Bae beach in Trat Province, Thailand. A VisitScotland spokeswoman said that "I'm sure it is a compliment in disguise."

The traditional Gaelic song 'Fàgail Bheàrnaraigh' (Leaving Berneray) later became the American folk song "O, Waly, Waly' The Water is Wide" with new lyrics added in English.

The English singer Vashti Bunyan wrote much of her debut album Just Another Diamond Day while living in a croft on Berneray. She revisits her old home at the end of Kieran Evans' documentary film about Bunyan's early life, Vashti Bunyan: From Here To Before.

Famous residents

 * Berneray is the birthplace of the giant Angus MacAskill.
 * Iain Eairdsidh MacAsgaill, a World War I veteran of the King's Own Cameron Highlanders, Western Australian bush poet in Scottish Gaelic, and highly important figure in 20th-century Scottish Gaelic literature, was born and raised on Berneray.
 * Donald MacKillop (1926 - 2015), a songwriter, poet, and important figure in Scottish Gaelic literature, was also from Berneray.
 * Fish (singer) (1958 -) the Scottish writer, singer & actor owns a crofting smallholding on Berneray and has publicised plans to permanently relocate there in 2024/5 with his partner, as he retires from the music industry.

title: Berneray (North Uist)


text:
Comma-separated values

Comma-separated values (CSV) is a text file format that uses commas to separate values, and newlines to separate records. A CSV file stores tabular data (numbers and text) in plain text, where each line of the file typically represents one data record. Each record consists of the same number of fields, and these are separated by commas in the CSV file. If the field delimiter itself may appear within a field, fields can be surrounded with quotation marks.

The CSV file format is one type of delimiter-separated file format. Delimiters frequently used include the comma, tab, space, and semicolon. Delimiter-separated files are often given a ".csv" extension even when the field separator is not a comma. Many applications or libraries that consume or produce CSV files have options to specify an alternative delimiter.

The lack of adherence to the CSV standard RFC 4180 necessitates the support for a variety of CSV formats in data input software. Despite this drawback, CSV remains widespread in data applications and is widely supported by a variety of software, including common spreadsheet applications such as Microsoft Excel. Benefits cited in favor of CSV include human readability and the simplicity of the format.

Applications
CSV is a common data exchange format that is widely supported by consumer, business, and scientific applications. Among its most common uses is moving tabular data between programs that natively operate on incompatible (often proprietary or undocumented) formats. For example, a user may need to transfer information from a database program that stores data in a proprietary format, to a spreadsheet that uses a completely different format. Most database programs can export data as CSV. Most spreadsheet programs can read CSV data, allowing CSV to be used as an intermediate format when transferring data from a database to a spreadsheet.

CSV is also used for storing data. Common data science tools such as Pandas include the option to export data to CSV for long-term storage. Benefits of CSV for data storage include the simplicity of CSV makes parsing and creating CSV files easy to implement and fast compared to other data formats, human readability making editing or fixing data simpler, and high compressibility leading to smaller data files. Alternatively, CSV does not support more complex data relations and makes no distinction between null and empty values, and in applications where these features are needed other formats are preferred.

Specification
proposes a specification for the CSV format; however, actual practice often does not follow the RFC and the term "CSV" might refer to any file that:

 * 1) is plain text using a character encoding such as ASCII, various Unicode character encodings (e.g. UTF-8), EBCDIC, or Shift JIS,
 * 2) consists of records (typically one record per line),
 * 3) with the records divided into fields separated by delimiters (typically a single reserved character such as comma, semicolon, or tab; sometimes the delimiter may include optional spaces),
 * 4) where every record has the same sequence of fields.

Within these general constraints, many variations are in use. Therefore, without additional information (such as whether RFC 4180 is honored), a file claimed simply to be in "CSV" format is not fully specified. As a result, some applications supporting CSV files have text import wizards that allow users to preview the first few lines of the file and then specify the delimiter character(s), quoting rules, and field trimming.

History
Comma-separated values is a data format that predates personal computers by more than a decade: the IBM Fortran (level H extended) compiler under OS/360 supported CSV in 1972. List-directed ("free form") input/output was defined in FORTRAN 77, approved in 1978. List-directed input used commas or spaces for delimiters, so unquoted character strings could not contain commas or spaces.

The term "comma-separated value" and the "CSV" abbreviation were in use by 1983. The manual for the Osborne Executive computer, which bundled the SuperCalc spreadsheet, documents the CSV quoting convention that allows strings to contain embedded commas, but the manual does not specify a convention for embedding quotation marks within quoted strings.

Comma-separated value lists are easier to type (for example into punched cards) than fixed-column-aligned data, and they were less prone to producing incorrect results if a value was punched one column off from its intended location.

Comma separated files are used for the interchange of database information between machines of two different architectures. The plain-text character of CSV files largely avoids incompatibilities such as byte-order and word size. The files are largely human-readable, so it is easier to deal with them in the absence of perfect documentation or communication.

The main standardization initiative—transforming "de facto fuzzy definition" into a more precise and de jure one—was in 2005, with, defining CSV as a MIME Content Type. Later, in 2013, some of RFC 4180's deficiencies were tackled by a W3C recommendation.

In 2014 IETF published describing the application of URI fragments to CSV documents. RFC 7111 specifies how row, column, and cell ranges can be selected from a CSV document using position indexes.

In 2015 W3C, in an attempt to enhance CSV with formal semantics, publicized the first drafts of recommendations for CSV metadata standards, which began as recommendations in December of the same year.

General functionality
CSV formats are best used to represent sets or sequences of records in which each record has an identical list of fields. This corresponds to a single relation in a relational database, or to data (though not calculations) in a typical spreadsheet.

The format dates back to the early days of business computing and is widely used to pass data between computers with different internal word sizes, data formatting needs, and so forth. For this reason, CSV files are common on all computer platforms.

CSV is a delimited text file that uses a comma to separate values (many implementations of CSV import/export tools allow other separators to be used; for example, the use of a "Sep=^" row as the first row in the *.csv file will cause Excel to open the file expecting caret "^" to be the separator instead of comma ","). Simple CSV implementations may prohibit field values that contain a comma or other special characters such as newlines. More sophisticated CSV implementations permit them, often by requiring " (double quote) characters around values that contain reserved characters (such as commas, double quotes, or less commonly, newlines). Embedded double quote characters may then be represented by a pair of consecutive double quotes, or by prefixing a double quote with an escape character such as a backslash (for example in Sybase Central).

CSV formats are not limited to a particular character set. They work just as well with Unicode character sets (such as UTF-8 or UTF-16) as with ASCII (although particular programs that support CSV may have their own limitations). CSV files normally will even survive naïve translation from one character set to another (unlike nearly all proprietary data formats). CSV does not, however, provide any way to indicate what character set is in use, so that must be communicated separately, or determined at the receiving end (if possible).

Databases that include multiple relations cannot be exported as a single CSV file. Similarly, CSV cannot naturally represent hierarchical or object-oriented data. This is because every CSV record is expected to have the same structure. CSV is therefore rarely appropriate for documents created with HTML, XML, or other markup or word-processing technologies.

Statistical databases in various fields often have a generally relation-like structure, but with some repeatable groups of fields. For example, health databases such as the Demographic and Health Survey typically repeat some questions for each child of a given parent (perhaps up to a fixed maximum number of children). Statistical analysis systems often include utilities that can "rotate" such data; for example, a "parent" record that includes information about five children can be split into five separate records, each containing (a) the information on one child, and (b) a copy of all the non-child-specific information. CSV can represent either the "vertical" or "horizontal" form of such data.

In a relational database, similar issues are readily handled by creating a separate relation for each such group, and connecting "child" records to the related "parent" records using a foreign key (such as an ID number or name for the parent). In markup languages such as XML, such groups are typically enclosed within a parent element and repeated as necessary (for example, multiple  nodes within a single   node). With CSV there is no widely accepted single-file solution.

Standardization
The name "CSV" indicates the use of the comma to separate data fields. Nevertheless, the term "CSV" is widely used to refer to a large family of formats that differ in many ways. Some implementations allow or require single or double quotation marks around some or all fields; and some reserve the first record as a header containing a list of field names. The character set being used is undefined: some applications require a Unicode byte order mark (BOM) to enforce Unicode interpretation (sometimes even a UTF-8 BOM). Files that use the tab character instead of comma can be more precisely referred to as "TSV" for tab-separated values.

Other implementation differences include the handling of more commonplace field separators (such as space or semicolon) and newline characters inside text fields. One more subtlety is the interpretation of a blank line: it can equally be the result of writing a record of zero fields, or a record of one field of zero length; thus decoding it is ambiguous.

RFC 4180 and MIME standards
The 2005 technical standard RFC 4180 formalizes the CSV file format and defines the MIME type "text/csv" for the handling of text-based fields. However, the interpretation of the text of each field is still application-specific. Files that follow the RFC 4180 standard can simplify CSV exchange and should be widely portable. Among its requirements:
 * MS-DOS-style lines that end with (CR/LF) characters (optional for the last line).
 * An optional header record (there is no sure way to detect whether it is present, so care is required when importing).
 * Each record should contain the same number of comma-separated fields.
 * Any field may be quoted (with double quotes).
 * Fields containing a line-break, double-quote or commas should be quoted. (If they are not, the file will likely be impossible to process correctly.)
 * If double-quotes are used to enclose fields, then a double-quote in a field must be represented by two double-quote characters.

The format can be processed by most programs that claim to read CSV files. The exceptions are (a) programs may not support line-breaks within quoted fields, (b) programs may confuse the optional header with data or interpret the first data line as an optional header, and (c) double-quotes in a field may not be parsed correctly automatically.

OKF frictionless tabular data package
In 2011 Open Knowledge Foundation (OKF) and various partners created a data protocols working group, which later evolved into the Frictionless Data initiative. One of the main formats they released was the Tabular Data Package. Tabular Data package was heavily based on CSV, using it as the main data transport format and adding basic type and schema metadata (CSV lacks any type information to distinguish the string "1" from the number 1).

The Frictionless Data Initiative has also provided a standard CSV Dialect Description Format for describing different dialects of CSV, for example specifying the field separator or quoting rules.

W3C tabular data standard
In 2013 the W3C "CSV on the Web" working group began to specify technologies providing higher interoperability for web applications using CSV or similar formats. The working group completed its work in February 2016 and is officially closed in March 2016 with the release of a set of documents and W3C recommendations for modeling "Tabular Data", and enhancing CSV with metadata and semantics.

Basic rules
Many informal documents exist that describe "CSV" formats. IETF RFC 4180 (summarized above) defines the format for the "text/csv" MIME type registered with the IANA.

Rules typical of these and other "CSV" specifications and implementations are as follows:

Example
The above table of data may be represented in CSV format as follows:

Year,Make,Model,Description,Price 1997,Ford,E350,"ac, abs, moon",3000.00 1999,Chevy,"Venture ""Extended Edition""","",4900.00 1999,Chevy,"Venture ""Extended Edition, Very Large""","",5000.00 1996,Jeep,Grand Cherokee,"MUST SELL! air, moon roof, loaded",4799.00

Example of a USA/UK CSV file (where the decimal separator is a period/full stop and the value separator is a comma):

Year,Make,Model,Length 1997,Ford,E350,2.35 2000,Mercury,Cougar,2.38

Example of an analogous European CSV/DSV file (where the decimal separator is a comma and the value separator is a semicolon):

Year;Make;Model;Length 1997;Ford;E350;2,35 2000;Mercury;Cougar;2,38

The latter format is not RFC 4180 compliant. Compliance could be achieved by the use of a comma instead of a semicolon as a separator and by quoting all numbers that have a decimal mark.

Application support
Some applications use CSV as a data interchange format to enhance its interoperability, exporting and importing CSV. Others use CSV as an internal format.

As a data interchange format: the CSV file format is supported by almost all spreadsheets and database management systems,
 * Spreadsheets including Apple Numbers, LibreOffice Calc, and Apache OpenOffice Calc. Microsoft Excel also supports a dialect of CSV with restrictions in comparison to other spreadsheet software (e.g., Excel still cannot export CSV files in the commonly used UTF-8 character encoding, and separator is not enforced to be the comma). LibreOffice Calc CSV importer is actually a more generic delimited text importer, supporting multiple separators at the same time as well as field trimming.
 * Various Relational databases support saving query results to a CSV file. PostgreSQL provides  command, which allows for both saving and loading data to and from a file. COPY (SELECT * FROM articles) TO '/home/wikipedia/file.csv' (FORMAT csv) saves the content of a table   to a file called.
 * Many utility programs on Unix-style systems (such as cut, paste, join, sort, uniq, awk) can split files on a comma delimiter, and can therefore process simple CSV files. However, this method does not correctly handle commas or new lines within quoted strings.

As (main or optional) internal representation. Can be native or foreign, but differ from interchange format ("export/import only") because it is not necessary to create a copy in another format:
 * Some Spreadsheets including LibreOffice Calc offers this option, without enforcing user to adopt another format.
 * Some relational databases, when using standard SQL, offer foreign-data wrapper (FDW). For example, PostgreSQL offers the CREATE FOREIGN TABLE and CREATE EXTENSION file_fdw to configure any variant of CSV.
 * Databases like Apache Hive offer the option to express CSV or .csv.gz as an internal table format.
 * The emacs editor can operate on CSV files using csv-nav mode.

CSV format is supported by libraries available for many programming languages. Most provide some way to specify the field delimiter, decimal separator, character encoding, quoting conventions, date format, etc.

Software and row limits
Programs that work with CSV may have limits on the maximum number of rows CSV files can have. Below is a list of common software and its limitations:
 * Microsoft Excel: 1,048,576 row limit;
 * Apple Numbers: 1,000,000 row limit;
 * Google Sheets: 5,000,000 cell limit (the product of columns and rows);
 * OpenOffice and LibreOffice: 1,048,576 row limit;
 * Text Editors (such as WordPad, TextEdit, Vim, etc.): no row or cell limit;
 * Databases (COPY command and FDW): no row or cell limit.

title: Comma-separated values


text:
Max More

Max More (born Max T. O'Connor, January 1964, with name legally changed in 1990) is a philosopher and futurist who writes, speaks, and consults on emerging technologies. He was the president and CEO of the Alcor Life Extension Foundation between 2010 and 2020.

Born in Bristol, England, More has a degree in Philosophy, Politics and Economics from St Anne's College, Oxford (1987). His 1995 University of Southern California doctoral dissertation The Diachronic Self: Identity, Continuity, and Transformation examined several issues that concern transhumanists, including the nature of death, and what it is about each individual that continues despite great change over time. In 1996, he married transhumanist Natasha Vita-More; the couple are close collaborators on transhumanist and life extension research.

More founded the Extropy Institute and has written many articles espousing the philosophy of transhumanism and the transhumanist philosophy of extropianism, including his "Principles of Extropy". In a 1990 essay "Transhumanism: Toward a Futurist Philosophy", he introduced the term "transhumanism" in its modern sense.

title: Max More


text:
General National Vocational Qualification

A General National Vocational Qualification, or GNVQ, was a certificate of vocational education in the United Kingdom. The last GNVQs were awarded in 2007.

The qualifications related to occupational areas in general, rather than any specific job. They could be taken in a wide range of subjects. There were different levels of GNVQ, namely the Intermediate level (equivalent to four General Certificates of Secondary Education) and Advanced level (equivalent to two Advanced-level General Certificates of Education). GNVQs were available to people of all ages. Many schools and colleges offered these courses and they could be studied alongside GCSEs or A levels. The GNVQ generally involved much coursework (6-8 large assignments), which allowed holders to show their skills when applying for jobs.

GNVQs were used in many schools in Wales in three main subjects: Engineering, Health and Social Care, and Leisure and Tourism. They were commonly used between early 2000 and 2005 as one-year courses for post-GCSE students who wished to further their education within the sixth-form but not at an A-Level standard. The course was generally undertaken in Wales at Intermediate level with day-release at an appropriate work placement during school hours for the duration of the year. Subsequent to 2005 many schools stopped using GNVQ courses in the sixth form and after 2004 incorporated Intermediate or Foundation level GNVQs into the syllabus in years 10 and 11 along with GCSEs. This was to encourage more students to undertake vocational areas of study as part of their compulsory education. The majority of one-year courses for sixth formers were then replaced with BTEC First Diplomas and Certificates.

GNVQs ceased to be offered in late 2007. Pupils in England and Wales are able to undertake BTEC and OCR national certificates and diplomas instead. Alternatives to GNVQ qualifications include vocational GCSEs, BTEC diplomas and certificates, OCR Nationals and City and Guild progression awards. The Diploma in Digital Applications, launched in 2005, was seen as a successor to GNVQ ICT, however many centres quickly switched to the equivalent OCR National qualification due to issues with DiDA's assessment format.


text:
Baleshare

Baleshare (Baile Sear) is a flat tidal island in the Outer Hebrides of Scotland. Baleshare lies to the south-west of North Uist. Its economics and community were boosted by the building of a causeway in 1962. The 350 m causeway was built by William Tawse Ltd. The island is extremely flat by Hebridean standards, rising to only 12 m above sea level and known for its long sandy beach. It has a population of 49 living in two settlements: Samhla and Teananachar.

Place names and locations
The name means "east farm" or "east town". The "west town" may have been on land that was said to exist to the west of Baleshare, washed away in the sixteenth century, over which it was possible to walk to the Monach Islands at low tide. The Monachs are some 15 km to the west. Another story suggests there was once a land bridge to Kirkibost, 100 m to the north. The basis for this seems to be a reference in the Exchequer Rolls for 1542 that the valued rental had been decreased due to encroachment by the sea at some unspecified (presumably recent) date.

Archaeology
Two prehistoric settlements have been uncovered, which contain the remains of a circular stone house and pieces of pottery, bone and metal. In common with other sites in the area, they are threatened by coastal erosion.

Baelshare was chosen as one of the pilot projects for the Shorewatch programme, which aimed to training community groups to search for new archaeological sites and record information on them to be passed on to local and national archives. In the first year of the pilot project, 2005, the island was severely affected by storms. In some cases, up to 50 metres of the coast were lost in a single night. Between August and December 2005, up to four metres of archaeological remains were lost to erosion.

Notable people from Baleshare

 * John Fergusson, Nova Scotia politician.

title: Baleshare