text:
Chain rule

In calculus, the chain rule is a formula that expresses the derivative of the composition of two differentiable functions $f$ and $g$ in terms of the derivatives of $f$ and $g$. More precisely, if $$h=f\circ g$$ is the function such that $$h(x)=f(g(x))$$ for every $x$, then the chain rule is, in Lagrange's notation, $$h'(x) = f'(g(x)) g'(x).$$ or, equivalently, $$h'=(f\circ g)'=(f'\circ g)\cdot g'.$$

The chain rule may also be expressed in Leibniz's notation. If a variable $z$ depends on the variable $y$, which itself depends on the variable $x$ (that is, $y$ and $z$ are dependent variables), then $z$ depends on $x$ as well, via the intermediate variable $y$. In this case, the chain rule is expressed as $$\frac{dz}{dx} = \frac{dz}{dy} \cdot \frac{dy}{dx},$$ and $$ \left.\frac{dz}{dx}\right|_{x} = \left.\frac{dz}{dy}\right|_{y(x)} \cdot \left. \frac{dy}{dx}\right|_{x} ,$$ for indicating at which points the derivatives have to be evaluated.

In integration, the counterpart to the chain rule is the substitution rule.

Intuitive explanation
Intuitively, the chain rule states that knowing the instantaneous rate of change of $z$ relative to $y$ and that of $y$ relative to $x$ allows one to calculate the instantaneous rate of change of $z$ relative to $x$ as the product of the two rates of change.

As put by George F. Simmons: "If a car travels twice as fast as a bicycle and the bicycle is four times as fast as a walking man, then the car travels 2 × 4 = 8 times as fast as the man."

The relationship between this example and the chain rule is as follows. Let $z$, $y$ and $x$ be the (variable) positions of the car, the bicycle, and the walking man, respectively. The rate of change of relative positions of the car and the bicycle is Similarly,  So, the rate of change of the relative positions of the car and the walking man is $$\frac{dz}{dx}=\frac{dz}{dy}\cdot\frac{dy}{dx}=2\cdot 4=8.$$

The rate of change of positions is the ratio of the speeds, and the speed is the derivative of the position with respect to the time; that is, $$\frac{dz}{dx}=\frac \frac{dz}{dt}\frac{dx}{dt},$$ or, equivalently, $$\frac{dz}{dt}=\frac{dz}{dx}\cdot \frac{dx}{dt},$$ which is also an application of the chain rule.

History
The chain rule seems to have first been used by Gottfried Wilhelm Leibniz. He used it to calculate the derivative of $$\sqrt{a + bz + cz^2}$$ as the composite of the square root function and the function $$a + bz + cz^2\!$$. He first mentioned it in a 1676 memoir (with a sign error in the calculation). The common notation of the chain rule is due to Leibniz. Guillaume de l'Hôpital used the chain rule implicitly in his Analyse des infiniment petits. The chain rule does not appear in any of Leonhard Euler's analysis books, even though they were written over a hundred years after Leibniz's discovery.. It is believed that the first "modern" version of the chain rule appears in Lagrange’s 1797 Théorie des fonctions analytiques; it also appears in Cauchy’s 1823 Résumé des Leçons données a L’École Royale Polytechnique sur Le Calcul Infinitesimal.

Statement
The simplest form of the chain rule is for real-valued functions of one real variable. It states that if $g$ is a function that is differentiable at a point $c$ (i.e. the derivative $g′(c)$ exists) and $f$ is a function that is differentiable at $g(c)$, then the composite function $$f \circ g$$ is differentiable at $c$, and the derivative is $$ (f\circ g)'(c) = f'(g(c))\cdot g'(c). $$ The rule is sometimes abbreviated as $$(f\circ g)' = (f'\circ g) \cdot g'.$$

If $y = f(u)$ and $u = g(x)$, then this abbreviated form is written in Leibniz notation as: $$\frac{dy}{dx} = \frac{dy}{du} \cdot \frac{du}{dx}.$$

The points where the derivatives are evaluated may also be stated explicitly: $$\left.\frac{dy}{dx}\right|_{x=c} = \left.\frac{dy}{du}\right|_{u = g(c)} \cdot \left.\frac{du}{dx}\right|_{x=c}.$$

Carrying the same reasoning further, given $n$ functions $$f_1, \ldots, f_n\!$$ with the composite function $$f_1 \circ ( f_2 \circ \cdots (f_{n-1} \circ f_n) )\!$$, if each function $$f_i\!$$ is differentiable at its immediate input, then the composite function is also differentiable by the repeated application of Chain Rule, where the derivative is (in Leibniz's notation): $$\frac{df_1}{dx} = \frac{df_1}{df_2}\frac{df_2}{df_3}\cdots\frac{df_n}{dx}.$$

Composites of more than two functions
The chain rule can be applied to composites of more than two functions. To take the derivative of a composite of more than two functions, notice that the composite of $f$, $g$, and $h$ (in that order) is the composite of $f$ with $g ∘ h$. The chain rule states that to compute the derivative of $f ∘ g ∘ h$, it is sufficient to compute the derivative of $f$ and the derivative of $g ∘ h$. The derivative of $f$ can be calculated directly, and the derivative of $g ∘ h$ can be calculated by applying the chain rule again.

For concreteness, consider the function $$y = e^{\sin (x^2)}.$$ This can be decomposed as the composite of three functions: $$\begin{align} y &= f(u) = e^u, \\[6pt] u &= g(v) = \sin v, \\[6pt] v &= h(x) = x^2. \end{align}$$ So that $$ y = f(g(h(x))) $$.

Their derivatives are: $$\begin{align} \frac{dy}{du} &= f'(u) = e^u, \\[6pt] \frac{du}{dv} &= g'(v) = \cos v, \\[6pt] \frac{dv}{dx} &= h'(x) = 2x. \end{align}$$

The chain rule states that the derivative of their composite at the point $x = a$ is: $$\begin{align} (f \circ g \circ h)'(a) & = f'((g \circ h)(a)) \cdot (g \circ h)'(a) \\[10pt] & = f'((g \circ h)(a)) \cdot g'(h(a)) \cdot h'(a) \\[10pt] & = (f' \circ g \circ h)(a) \cdot (g' \circ h)(a) \cdot h'(a). \end{align}$$

In Leibniz's notation, this is: $$\frac{dy}{dx} = \left.\frac{dy}{du}\right|_{u=g(h(a))}\cdot\left.\frac{du}{dv}\right|_{v=h(a)}\cdot\left.\frac{dv}{dx}\right|_{x=a},$$ or for short, $$\frac{dy}{dx} = \frac{dy}{du}\cdot\frac{du}{dv}\cdot\frac{dv}{dx}.$$ The derivative function is therefore: $$\frac{dy}{dx} = e^{\sin(x^2)}\cdot\cos(x^2)\cdot 2x.$$

Another way of computing this derivative is to view the composite function $f ∘ g ∘ h$ as the composite of $f ∘ g$ and h. Applying the chain rule in this manner would yield: $$(f \circ g \circ h)'(a) = (f \circ g)'(h(a))\cdot h'(a) = f'(g(h(a)))\cdot g'(h(a))\cdot h'(a).$$

This is the same as what was computed above. This should be expected because $(f ∘ g) ∘ h = f ∘ (g ∘ h)$.

Sometimes, it is necessary to differentiate an arbitrarily long composition of the form $$f_1 \circ f_2 \circ \cdots \circ f_{n-1} \circ f_n\!$$. In this case, define $$f_{a\,.\,.\,b} = f_{a} \circ f_{a+1} \circ \cdots \circ f_{b-1} \circ f_{b}$$ where $$f_{a\,.\,.\,a} = f_a$$ and $$f_{a\,.\,.\,b}(x) = x$$ when $$b < a$$. Then the chain rule takes the form $$Df_{1\,.\,.\,n} = (Df_1 \circ f_{2\,.\,.\,n}) (Df_2 \circ f_{3\,.\,.\,n}) \cdots (Df_{n-1} \circ f_{n\,.\,.\,n}) Df_n = \prod_{k=1}^n \left[Df_k \circ f_{(k+1)\,.\,.\,n}\right]$$ or, in the Lagrange notation, $$f_{1\,.\,.\,n}'(x) = f_1' \left( f_{2\,.\,.\,n}(x) \right) \; f_2' \left( f_{3\,.\,.\,n}(x) \right) \cdots f_{n-1}' \left(f_{n\,.\,.\,n}(x)\right) \; f_n'(x) = \prod_{k=1}^{n} f_k' \left(f_{(k+1\,.\,.\,n)}(x) \right)$$

Quotient rule
The chain rule can be used to derive some well-known differentiation rules. For example, the quotient rule is a consequence of the chain rule and the product rule. To see this, write the function $f(x)/g(x)$ as the product $f(x) · 1/g(x)$. First apply the product rule: $$\begin{align} \frac{d}{dx}\left(\frac{f(x)}{g(x)}\right) &= \frac{d}{dx}\left(f(x)\cdot\frac{1}{g(x)}\right) \\ &= f'(x)\cdot\frac{1}{g(x)} + f(x)\cdot\frac{d}{dx}\left(\frac{1}{g(x)}\right). \end{align}$$

To compute the derivative of $1/g(x)$, notice that it is the composite of $g$ with the reciprocal function, that is, the function that sends $x$ to $1/x$. The derivative of the reciprocal function is $$-1/x^2\!$$. By applying the chain rule, the last expression becomes: $$f'(x)\cdot\frac{1}{g(x)} + f(x)\cdot\left(-\frac{1}{g(x)^2}\cdot g'(x)\right) = \frac{f'(x) g(x) - f(x) g'(x)}{g(x)^2},$$ which is the usual formula for the quotient rule.

Derivatives of inverse functions
Suppose that $y = g(x)$ has an inverse function. Call its inverse function $f$ so that we have $x = f(y)$. There is a formula for the derivative of $f$ in terms of the derivative of $g$. To see this, note that $f$ and $g$ satisfy the formula $$f(g(x)) = x.$$

And because the functions $$f(g(x))$$ and $x$ are equal, their derivatives must be equal. The derivative of $x$ is the constant function with value 1, and the derivative of $$f(g(x))$$ is determined by the chain rule. Therefore, we have that: $$f'(g(x)) g'(x) = 1.$$

To express $f'$ as a function of an independent variable $y$, we substitute $$f(y)$$ for $x$ wherever it appears. Then we can solve for $f'$. $$\begin{align} f'(g(f(y))) g'(f(y)) &= 1 \\[5pt] f'(y) g'(f(y)) &= 1 \\[5pt] f'(y) = \frac{1}{g'(f(y))}. \end{align}$$

For example, consider the function $g(x) = e^{x}$. It has an inverse $f(y) = ln y$. Because $g′(x) = e^{x}$, the above formula says that $$\frac{d}{dy}\ln y = \frac{1}{e^{\ln y}} = \frac{1}{y}.$$

This formula is true whenever $g$ is differentiable and its inverse $f$ is also differentiable. This formula can fail when one of these conditions is not true. For example, consider $g(x) = x^{3}$. Its inverse is $f(y) = y^{1/3}$, which is not differentiable at zero. If we attempt to use the above formula to compute the derivative of $f$ at zero, then we must evaluate $1/g′(f(0))$. Since $f(0) = 0$ and $g′(0) = 0$, we must evaluate 1/0, which is undefined. Therefore, the formula fails in this case. This is not surprising because $f$ is not differentiable at zero.

Back propagation
The chain rule forms the basis of the back propagation algorithm, which is used in gradient descent of neural networks in deep learning (artificial intelligence).

Higher derivatives
Faà di Bruno's formula generalizes the chain rule to higher derivatives. Assuming that $y = f(u)$ and $u = g(x)$, then the first few derivatives are: $$ \begin{align} \frac{dy}{dx} & = \frac{dy}{du} \frac{du}{dx} \\[4pt] \frac{d^2 y }{d x^2} & = \frac{d^2 y}{d u^2} \left(\frac{du}{dx}\right)^2 + \frac{dy}{du} \frac{d^2 u}{dx^2} \\[4pt] \frac{d^3 y }{d x^3} & = \frac{d^3 y}{d u^3} \left(\frac{du}{dx}\right)^3 + 3 \, \frac{d^2 y}{d u^2} \frac{du}{dx} \frac{d^2 u}{d x^2} + \frac{dy}{du} \frac{d^3 u}{d x^3} \\[4pt] \frac{d^4 y}{d x^4} & = \frac{d^4 y}{du^4} \left(\frac{du}{dx}\right)^4 + 6 \, \frac{d^3 y}{d u^3} \left(\frac{du}{dx}\right)^2 \frac{d^2 u}{d x^2} + \frac{d^2 y}{d u^2} \left( 4 \, \frac{du}{dx} \frac{d^3 u}{dx^3}   + 3 \, \left(\frac{d^2 u}{dx^2}\right)^2\right) + \frac{dy}{du} \frac{d^4 u}{dx^4}. \end{align}$$

First proof
One proof of the chain rule begins by defining the derivative of the composite function $f ∘ g$, where we take the limit of the difference quotient for $f ∘ g$ as $x$ approaches $a$: $$(f \circ g)'(a) = \lim_{x \to a} \frac{f(g(x)) - f(g(a))}{x - a}.$$

Assume for the moment that $$g(x)\!$$ does not equal $$g(a)$$ for any $$x$$ near $$a$$. Then the previous expression is equal to the product of two factors: $$\lim_{x \to a} \frac{f(g(x)) - f(g(a))}{g(x) - g(a)} \cdot \frac{g(x) - g(a)}{x - a}.$$

If $$g$$ oscillates near $a$, then it might happen that no matter how close one gets to $a$, there is always an even closer $x$ such that $g(x) = g(a)$. For example, this happens near $a = 0$ for the continuous function $g$ defined by $g(x) = 0$ for $x = 0$ and $g(x) = x^{2} sin(1/x)$ otherwise. Whenever this happens, the above expression is undefined because it involves division by zero. To work around this, introduce a function $$Q$$ as follows: $$Q(y) = \begin{cases} \displaystyle\frac{f(y) - f(g(a))}{y - g(a)}, & y \neq g(a), \\ f'(g(a)), & y = g(a). \end{cases}$$ We will show that the difference quotient for $f ∘ g$ is always equal to: $$Q(g(x)) \cdot \frac{g(x) - g(a)}{x - a}.$$

Whenever $g(x)$ is not equal to $g(a)$, this is clear because the factors of $g(x) − g(a)$ cancel. When $g(x)$ equals $g(a)$, then the difference quotient for $f ∘ g$ is zero because $f(g(x))$ equals $f(g(a))$, and the above product is zero because it equals $f′(g(a))$ times zero. So the above product is always equal to the difference quotient, and to show that the derivative of $f ∘ g$ at $a$ exists and to determine its value, we need only show that the limit as $x$ goes to $a$ of the above product exists and determine its value.

To do this, recall that the limit of a product exists if the limits of its factors exist. When this happens, the limit of the product of these two factors will equal the product of the limits of the factors. The two factors are $Q(g(x))$ and $(g(x) − g(a)) / (x − a)$. The latter is the difference quotient for $g$ at $a$, and because $g$ is differentiable at $a$ by assumption, its limit as $x$ tends to $a$ exists and equals $g′(a)$.

As for $Q(g(x))$, notice that $Q$ is defined wherever $f$ is. Furthermore, $f$ is differentiable at $g(a)$ by assumption, so $Q$ is continuous at $g(a)$, by definition of the derivative. The function $g$ is continuous at $a$ because it is differentiable at $a$, and therefore $Q ∘ g$ is continuous at $a$. So its limit as $x$ goes to $a$ exists and equals $Q(g(a))$, which is $f′(g(a))$.

This shows that the limits of both factors exist and that they equal $f′(g(a))$ and $g′(a)$, respectively. Therefore, the derivative of $f ∘ g$ at a exists and equals $f′(g(a))$$g′(a)$.

Second proof
Another way of proving the chain rule is to measure the error in the linear approximation determined by the derivative. This proof has the advantage that it generalizes to several variables. It relies on the following equivalent definition of differentiability at a point: A function g is differentiable at a if there exists a real number g′(a) and a function ε(h) that tends to zero as h tends to zero, and furthermore $$g(a + h) - g(a) = g'(a) h + \varepsilon(h) h.$$ Here the left-hand side represents the true difference between the value of g at a and at $a + h$, whereas the right-hand side represents the approximation determined by the derivative plus an error term.

In the situation of the chain rule, such a function ε exists because g is assumed to be differentiable at a. Again by assumption, a similar function also exists for f at g(a). Calling this function η, we have $$f(g(a) + k) - f(g(a)) = f'(g(a)) k + \eta(k) k.$$ The above definition imposes no constraints on η(0), even though it is assumed that η(k) tends to zero as k tends to zero. If we set $η(0) = 0$, then η is continuous at 0.

Proving the theorem requires studying the difference $f(g(a + h)) − f(g(a))$ as h tends to zero. The first step is to substitute for $g(a + h)$ using the definition of differentiability of g at a: $$f(g(a + h)) - f(g(a)) = f(g(a) + g'(a) h + \varepsilon(h) h) - f(g(a)).$$ The next step is to use the definition of differentiability of f at g(a). This requires a term of the form $f(g(a) + k)$ for some k. In the above equation, the correct k varies with h. Set $k_{h} = g′(a) h + ε(h) h$ and the right hand side becomes $f(g(a) + k_{h}) − f(g(a))$. Applying the definition of the derivative gives: $$f(g(a) + k_h) - f(g(a)) = f'(g(a)) k_h + \eta(k_h) k_h.$$ To study the behavior of this expression as h tends to zero, expand kh. After regrouping the terms, the right-hand side becomes: $$f'(g(a)) g'(a)h + [f'(g(a)) \varepsilon(h) + \eta(k_h) g'(a) + \eta(k_h) \varepsilon(h)] h.$$ Because ε(h) and η(kh) tend to zero as h tends to zero, the first two bracketed terms tend to zero as h tends to zero. Applying the same theorem on products of limits as in the first proof, the third bracketed term also tends zero. Because the above expression is equal to the difference $f(g(a + h)) − f(g(a))$, by the definition of the derivative $f ∘ g$ is differentiable at a and its derivative is $f′(g(a)) g′(a).$

The role of Q in the first proof is played by η in this proof. They are related by the equation: $$Q(y) = f'(g(a)) + \eta(y - g(a)). $$ The need to define Q at g(a) is analogous to the need to define η at zero.

Third proof
Constantin Carathéodory's alternative definition of the differentiability of a function can be used to give an elegant proof of the chain rule.

Under this definition, a function $f$ is differentiable at a point $a$ if and only if there is a function $q$, continuous at $a$ and such that $f(x) − f(a) = q(x)(x − a)$. There is at most one such function, and if $f$ is differentiable at $a$ then $f ′(a) = q(a)$.

Given the assumptions of the chain rule and the fact that differentiable functions and compositions of continuous functions are continuous, we have that there exist functions $q$, continuous at $g(a)$, and $r$, continuous at $a$, and such that, $$f(g(x))-f(g(a))=q(g(x))(g(x)-g(a))$$ and $$g(x)-g(a)=r(x)(x-a).$$ Therefore, $$f(g(x))-f(g(a))=q(g(x))r(x)(x-a),$$ but the function given by $h(x) = q(g(x))r(x)$ is continuous at $a$, and we get, for this $a$ $$(f(g(a)))'=q(g(a))r(a)=f'(g(a))g'(a).$$ A similar approach works for continuously differentiable (vector-)functions of many variables. This method of factoring also allows a unified approach to stronger forms of differentiability, when the derivative is required to be Lipschitz continuous, Hölder continuous, etc. Differentiation itself can be viewed as the polynomial remainder theorem (the little Bézout theorem, or factor theorem), generalized to an appropriate class of functions.

Proof via infinitesimals
If $$y=f(x)$$ and $$x=g(t)$$ then choosing infinitesimal $$\Delta t\not=0$$ we compute the corresponding $$\Delta x=g(t+\Delta t)-g(t)$$ and then the corresponding $$\Delta y=f(x+\Delta x)-f(x)$$, so that $$\frac{\Delta y}{\Delta t} = \frac{\Delta y}{\Delta x} \frac{\Delta x}{\Delta t}$$ and applying the standard part we obtain $$\frac{d y}{d t}=\frac{d y}{d x} \frac{dx}{dt}$$ which is the chain rule.

Multivariable case
The full generalization of the chain rule to multi-variable functions (such as $$f : \mathbb{R}^m \to \mathbb{R}^n$$) is rather technical. However, it is simpler to write in the case of functions of the form $$f(g_1(x), \dots, g_k(x)),$$ where $$f : \reals^k \to \reals$$, and $$g_i : \mathbb{R} \to \mathbb{R}$$ for each $$i = 1, 2, \dots, k.$$

As this case occurs often in the study of functions of a single variable, it is worth describing it separately.

Case of scalar-valued functions with multiple inputs
Let $$f : \reals^k \to \reals$$, and $$g_i : \mathbb{R} \to \mathbb{R}$$ for each $$i = 1, 2, \dots, k.$$ To write the chain rule for the composition of functions $$x \mapsto f(g_1(x), \dots, g_k(x)),$$ one needs the partial derivatives of $f$ with respect to its $k$ arguments. The usual notations for partial derivatives involve names for the arguments of the function. As these arguments are not named in the above formula, it is simpler and clearer to use D-Notation, and to denote by $$D_i f$$ the partial derivative of $f$ with respect to its $i$th argument, and by $$D_i f(z)$$ the value of this derivative at $z$.

With this notation, the chain rule is $$\frac{d}{dx}f(g_1(x), \dots, g_k (x))=\sum_{i=1}^k \left(\frac{d}{dx}{g_i}(x)\right) D_i f(g_1(x), \dots, g_k (x)).$$

Example: arithmetic operations
If the function $f$ is addition, that is, if $$f(u,v)=u+v,$$ then $D_1 f = \frac{\partial f}{\partial u} = 1$ and $D_2 f = \frac{\partial f}{\partial v} = 1$. Thus, the chain rule gives $$\frac{d}{dx}(g(x)+h(x)) = \left( \frac{d}{dx}g(x) \right) D_1 f+\left( \frac{d}{dx}h(x)\right) D_2 f=\frac{d}{dx}g(x) +\frac{d}{dx}h(x).$$

For multiplication $$f(u,v)=uv,$$ the partials are $$D_1 f = v$$ and $$D_2 f = u$$. Thus, $$\frac{d}{dx}(g(x)h(x)) = h(x) \frac{d}{dx} g(x) + g(x) \frac{d}{dx} h(x).$$

The case of exponentiation $$f(u,v)=u^v$$ is slightly more complicated, as $$D_1 f = vu^{v-1},$$ and, as $$u^v=e^{v\ln u},$$ $$D_2 f = u^v\ln u.$$ It follows that $$\frac{d}{dx}\left(g(x)^{h(x)}\right) = h(x)g(x)^{h(x)-1} \frac{d}{dx}g(x) + g(x)^{h(x)} \ln g(x) \,\frac{d}{dx}h(x).$$

General rule: Vector-valued functions with multiple inputs
The simplest way for writing the chain rule in the general case is to use the total derivative, which is a linear transformation that captures all directional derivatives in a single formula. Consider differentiable functions $f : R^{m} → R^{k}$ and $g : R^{n} → R^{m}$, and a point $a$ in $R^{n}$. Let $D_{a} g$ denote the total derivative of $g$ at $a$ and $D_{g(a)} f$ denote the total derivative of $f$ at $g(a)$. These two derivatives are linear transformations $R^{n} → R^{m}$ and $R^{m} → R^{k}$, respectively, so they can be composed. The chain rule for total derivatives is that their composite is the total derivative of $f ∘ g$ at $a$: $$D_{\mathbf{a}}(f \circ g) = D_{g(\mathbf{a})}f \circ D_{\mathbf{a}}g,$$ or for short, $$D(f \circ g) = Df \circ Dg.$$ The higher-dimensional chain rule can be proved using a technique similar to the second proof given above.

Because the total derivative is a linear transformation, the functions appearing in the formula can be rewritten as matrices. The matrix corresponding to a total derivative is called a Jacobian matrix, and the composite of two derivatives corresponds to the product of their Jacobian matrices. From this perspective the chain rule therefore says: $$J_{f \circ g}(\mathbf{a}) = J_{f}(g(\mathbf{a})) J_{g}(\mathbf{a}),$$ or for short, $$J_{f \circ g} = (J_f \circ g)J_g.$$

That is, the Jacobian of a composite function is the product of the Jacobians of the composed functions (evaluated at the appropriate points).

The higher-dimensional chain rule is a generalization of the one-dimensional chain rule. If $k$, $m$, and $n$ are 1, so that $f : R → R$ and $g : R → R$, then the Jacobian matrices of $f$ and $g$ are $1 × 1$. Specifically, they are: $$\begin{align} J_g(a) &= \begin{pmatrix} g'(a) \end{pmatrix}, \\ J_{f}(g(a)) &= \begin{pmatrix} f'(g(a)) \end{pmatrix}. \end{align}$$ The Jacobian of $f ∘ g$ is the product of these $1 × 1$ matrices, so it is $f′(g(a))⋅g′(a)$, as expected from the one-dimensional chain rule. In the language of linear transformations, $D_{a}(g)$ is the function which scales a vector by a factor of $g′(a)$ and $D_{g(a)}(f)$ is the function which scales a vector by a factor of $f′(g(a))$. The chain rule says that the composite of these two linear transformations is the linear transformation $D_{a}(f ∘ g)$, and therefore it is the function that scales a vector by $f′(g(a))⋅g′(a)$.

Another way of writing the chain rule is used when f and g are expressed in terms of their components as $y = f(u) = (f_{1}(u), …, f_{k}(u))$ and $u = g(x) = (g_{1}(x), …, g_{m}(x))$. In this case, the above rule for Jacobian matrices is usually written as: $$\frac{\partial(y_1, \ldots, y_k)}{\partial(x_1, \ldots, x_n)} = \frac{\partial(y_1, \ldots, y_k)}{\partial(u_1, \ldots, u_m)} \frac{\partial(u_1, \ldots, u_m)}{\partial(x_1, \ldots, x_n)}.$$

The chain rule for total derivatives implies a chain rule for partial derivatives. Recall that when the total derivative exists, the partial derivative in the $i$-th coordinate direction is found by multiplying the Jacobian matrix by the $i$-th basis vector. By doing this to the formula above, we find: $$\frac{\partial(y_1, \ldots, y_k)}{\partial x_i} = \frac{\partial(y_1, \ldots, y_k)}{\partial(u_1, \ldots, u_m)} \frac{\partial(u_1, \ldots, u_m)}{\partial x_i}.$$ Since the entries of the Jacobian matrix are partial derivatives, we may simplify the above formula to get: $$\frac{\partial(y_1, \ldots, y_k)}{\partial x_i} = \sum_{\ell = 1}^m \frac{\partial(y_1, \ldots, y_k)}{\partial u_\ell} \frac{\partial u_\ell}{\partial x_i}.$$ More conceptually, this rule expresses the fact that a change in the $x_{i}$ direction may change all of $g_{1}$ through $g_{m}$, and any of these changes may affect $f$.

In the special case where $k = 1$, so that $f$ is a real-valued function, then this formula simplifies even further: $$\frac{\partial y}{\partial x_i} = \sum_{\ell = 1}^m \frac{\partial y}{\partial u_\ell} \frac{\partial u_\ell}{\partial x_i}.$$ This can be rewritten as a dot product. Recalling that $u = (g_{1}, …, g_{m})$, the partial derivative $∂u / ∂x_{i}$ is also a vector, and the chain rule says that: $$\frac{\partial y}{\partial x_i} = \nabla y \cdot \frac{\partial \mathbf{u}}{\partial x_i}.$$

Example
Given $u(x, y) = x^{2} + 2y$ where $x(r, t) = r sin(t)$ and $y(r,t) = sin^{2}(t)$, determine the value of $∂u / ∂r$ and $∂u / ∂t$ using the chain rule. $$\frac{\partial u}{\partial r}=\frac{\partial u}{\partial x} \frac{\partial x}{\partial r}+\frac{\partial u}{\partial y} \frac{\partial y}{\partial r} = (2x)(\sin(t)) + (2)(0) = 2r \sin^2(t),$$ and $$\begin{align} \frac{\partial u}{\partial t} &= \frac{\partial u}{\partial x} \frac{\partial x}{\partial t}+\frac{\partial u}{\partial y} \frac{\partial y}{\partial t} \\ &= (2x)(r\cos(t)) + (2)(2\sin(t)\cos(t)) \\ &= (2r\sin(t))(r\cos(t)) + 4\sin(t)\cos(t) \\ &= 2(r^2 + 2) \sin(t)\cos(t) \\ &= (r^2 + 2) \sin(2t). \end{align}$$

Higher derivatives of multivariable functions
Faà di Bruno's formula for higher-order derivatives of single-variable functions generalizes to the multivariable case. If $y = f(u)$ is a function of $u = g(x)$ as above, then the second derivative of $f ∘ g$ is: $$\frac{\partial^2 y}{\partial x_i \partial x_j} = \sum_k \left(\frac{\partial y}{\partial u_k}\frac{\partial^2 u_k}{\partial x_i \partial x_j}\right) + \sum_{k, \ell} \left(\frac{\partial^2 y}{\partial u_k \partial u_\ell}\frac{\partial u_k}{\partial x_i}\frac{\partial u_\ell}{\partial x_j}\right).$$

Further generalizations
All extensions of calculus have a chain rule. In most of these, the formula remains the same, though the meaning of that formula may be vastly different.

One generalization is to manifolds. In this situation, the chain rule represents the fact that the derivative of $f ∘ g$ is the composite of the derivative of $f$ and the derivative of $g$. This theorem is an immediate consequence of the higher dimensional chain rule given above, and it has exactly the same formula.

The chain rule is also valid for Fréchet derivatives in Banach spaces. The same formula holds as before. This case and the previous one admit a simultaneous generalization to Banach manifolds.

In differential algebra, the derivative is interpreted as a morphism of modules of Kähler differentials. A ring homomorphism of commutative rings $f : R → S$ determines a morphism of Kähler differentials $Df : Ω_{R} → Ω_{S}$ which sends an element $dr$ to $d(f(r))$, the exterior differential of $f(r)$. The formula $D(f ∘ g) = Df ∘ Dg$ holds in this context as well.

The common feature of these examples is that they are expressions of the idea that the derivative is part of a functor. A functor is an operation on spaces and functions between them. It associates to each space a new space and to each function between two spaces a new function between the corresponding new spaces. In each of the above cases, the functor sends each space to its tangent bundle and it sends each function to its derivative. For example, in the manifold case, the derivative sends a $C^{r}$-manifold to a $C^{r−1}$-manifold (its tangent bundle) and a $C^{r}$-function to its total derivative. There is one requirement for this to be a functor, namely that the derivative of a composite must be the composite of the derivatives. This is exactly the formula $D(f ∘ g) = Df ∘ Dg$.

There are also chain rules in stochastic calculus. One of these, Itō's lemma, expresses the composite of an Itō process (or more generally a semimartingale) dXt with a twice-differentiable function f. In Itō's lemma, the derivative of the composite function depends not only on dXt and the derivative of f but also on the second derivative of f. The dependence on the second derivative is a consequence of the non-zero quadratic variation of the stochastic process, which broadly speaking means that the process can move up and down in a very rough way. This variant of the chain rule is not an example of a functor because the two functions being composed are of different types.

title: Chain rule


text:
P versus NP problem

The P versus NP problem is a major unsolved problem in theoretical computer science. Informally, it asks whether every problem whose solution can be quickly verified can also be quickly solved.

Here, quickly means an algorithm that solves the task and runs in polynomial time exists, meaning the task completion time varies as a polynomial function on the size of the input to the algorithm (as opposed to, say, exponential time). The general class of questions that some algorithm can answer in polynomial time is "P" or "class P". For some questions, there is no known way to find an answer quickly, but if provided with an answer, it can be verified quickly. The class of questions where an answer can be verified in polynomial time is NP, standing for "nondeterministic polynomial time".

An answer to the P versus NP question would determine whether problems that can be verified in polynomial time can also be solved in polynomial time. If P ≠ NP, which is widely believed, it would mean that there are problems in NP that are harder to compute than to verify: they could not be solved in polynomial time, but the answer could be verified in polynomial time.

The problem has been called the most important open problem in computer science. Aside from being an important problem in computational theory, a proof either way would have profound implications for mathematics, cryptography, algorithm research, artificial intelligence, game theory, multimedia processing, philosophy, economics and many other fields.

It is one of the seven Millennium Prize Problems selected by the Clay Mathematics Institute, each of which carries a US$1,000,000 prize for the first correct solution.

Example
In the game Sudoku, the player begins with a partially filled-in grid of numbers and attempts to complete the grid following the game's rules. Given an incomplete Sudoku grid, of any size, is there at least one legal solution? Proposed solutions are easily verified, and the time to check a solution grows slowly (polynomially) as the grid gets bigger. However, all known algorithms for finding solutions take, for difficult examples, time that grows exponentially as the grid gets bigger. So, Sudoku is in NP (quickly checkable) but does not seem to be in P (quickly solvable). Thousands of other problems seem similarly fast to check but slow to solve. Researchers have shown that many of the problems in NP have the extra property that a fast solution to any one of them could be used to build a quick solution to any other problem in NP, a property called NP-completeness. Decades of searching have not produced a fast solution to any of these problems, so most scientists suspect that these problems cannot be solved quickly; however, this is unproven.

History
The precise statement of the P versus NP problem was introduced in 1971 by Stephen Cook in his seminal paper "The complexity of theorem proving procedures" (and independently by Leonid Levin in 1973 ).

Although the P versus NP problem was formally defined in 1971, there were previous inklings of the problems involved, the difficulty of proof, and the potential consequences. In 1955, mathematician John Nash wrote a letter to the NSA, speculating that cracking a sufficiently complex code would require time exponential in the length of the key. If proved (and Nash was suitably skeptical), this would imply what is now called P ≠ NP, since a proposed key can be verified in polynomial time. Another mention of the underlying problem occurred in a 1956 letter written by Kurt Gödel to John von Neumann. Gödel asked whether theorem-proving (now known to be co-NP-complete) could be solved in quadratic or linear time, and pointed out one of the most important consequences—that if so, then the discovery of mathematical proofs could be automated.

Context
The relation between the complexity classes P and NP is studied in computational complexity theory, the part of the theory of computation dealing with the resources required during computation to solve a given problem. The most common resources are time (how many steps it takes to solve a problem) and space (how much memory it takes to solve a problem).

In such analysis, a model of the computer for which time must be analyzed is required. Typically such models assume that the computer is deterministic (given the computer's present state and any inputs, there is only one possible action that the computer might take) and sequential (it performs actions one after the other).

In this theory, the class P consists of all decision problems (defined below) solvable on a deterministic sequential machine in a duration polynomial in the size of the input; the class NP consists of all decision problems whose positive solutions are verifiable in polynomial time given the right information, or equivalently, whose solution can be found in polynomial time on a non-deterministic machine. Clearly, P ⊆ NP. Arguably, the biggest open question in theoretical computer science concerns the relationship between those two classes:
 * Is P equal to NP?

Since 2002, William Gasarch has conducted three polls of researchers concerning this and related questions. Confidence that P ≠ NP has been increasing – in 2019, 88% believed P ≠ NP, as opposed to 83% in 2012 and 61% in 2002. When restricted to experts, the 2019 answers became 99% believed P ≠ NP. These polls do not imply whether P = NP, Gasarch himself stated: "This does not bring us any closer to solving P=?NP or to knowing when it will be solved, but it attempts to be an objective report on the subjective opinion of this era."

NP-completeness

To attack the P = NP question, the concept of NP-completeness is very useful. NP-complete problems are problems that any other NP problem is reducible to in polynomial time and whose solution is still verifiable in polynomial time. That is, any NP problem can be transformed into any NP-complete problem. Informally, an NP-complete problem is an NP problem that is at least as "tough" as any other problem in NP.

NP-hard problems are those at least as hard as NP problems; i.e., all NP problems can be reduced (in polynomial time) to them. NP-hard problems need not be in NP; i.e., they need not have solutions verifiable in polynomial time.

For instance, the Boolean satisfiability problem is NP-complete by the Cook–Levin theorem, so any instance of any problem in NP can be transformed mechanically into a Boolean satisfiability problem in polynomial time. The Boolean satisfiability problem is one of many NP-complete problems. If any NP-complete problem is in P, then it would follow that P = NP. However, many important problems are NP-complete, and no fast algorithm for any of them is known.

From the definition alone it is unintuitive that NP-complete problems exist; however, a trivial NP-complete problem can be formulated as follows: given a Turing machine M guaranteed to halt in polynomial time, does a polynomial-size input that M will accept exist? It is in NP because (given an input) it is simple to check whether M accepts the input by simulating M; it is NP-complete because the verifier for any particular instance of a problem in NP can be encoded as a polynomial-time machine M that takes the solution to be verified as input. Then the question of whether the instance is a yes or no instance is determined by whether a valid input exists.

The first natural problem proven to be NP-complete was the Boolean satisfiability problem, also known as SAT. As noted above, this is the Cook–Levin theorem; its proof that satisfiability is NP-complete contains technical details about Turing machines as they relate to the definition of NP. However, after this problem was proved to be NP-complete, proof by reduction provided a simpler way to show that many other problems are also NP-complete, including the game Sudoku discussed earlier. In this case, the proof shows that a solution of Sudoku in polynomial time could also be used to complete Latin squares in polynomial time. This in turn gives a solution to the problem of partitioning tri-partite graphs into triangles, which could then be used to find solutions for the special case of SAT known as 3-SAT, which then provides a solution for general Boolean satisfiability. So a polynomial-time solution to Sudoku leads, by a series of mechanical transformations, to a polynomial time solution of satisfiability, which in turn can be used to solve any other NP-problem in polynomial time. Using transformations like this, a vast class of seemingly unrelated problems are all reducible to one another, and are in a sense "the same problem".

Harder problems
Although it is unknown whether P = NP, problems outside of P are known. Just as the class P is defined in terms of polynomial running time, the class EXPTIME is the set of all decision problems that have exponential running time. In other words, any problem in EXPTIME is solvable by a deterministic Turing machine in O(2p(n)) time, where p(n) is a polynomial function of n. A decision problem is EXPTIME-complete if it is in EXPTIME, and every problem in EXPTIME has a polynomial-time many-one reduction to it. A number of problems are known to be EXPTIME-complete. Because it can be shown that P ≠ EXPTIME, these problems are outside P, and so require more than polynomial time. In fact, by the time hierarchy theorem, they cannot be solved in significantly less than exponential time. Examples include finding a perfect strategy for chess positions on an N × N board and similar problems for other board games.

The problem of deciding the truth of a statement in Presburger arithmetic requires even more time. Fischer and Rabin proved in 1974 that every algorithm that decides the truth of Presburger statements of length n has a runtime of at least $$2^{2^{cn}}$$ for some constant c. Hence, the problem is known to need more than exponential run time. Even more difficult are the undecidable problems, such as the halting problem. They cannot be completely solved by any algorithm, in the sense that for any particular algorithm there is at least one input for which that algorithm will not produce the right answer; it will either produce the wrong answer, finish without giving a conclusive answer, or otherwise run forever without producing any answer at all.

It is also possible to consider questions other than decision problems. One such class, consisting of counting problems, is called #P: whereas an NP problem asks "Are there any solutions?", the corresponding #P problem asks "How many solutions are there?". Clearly, a #P problem must be at least as hard as the corresponding NP problem, since a count of solutions immediately tells if at least one solution exists, if the count is greater than zero. Surprisingly, some #P problems that are believed to be difficult correspond to easy (for example linear-time) P problems. For these problems, it is very easy to tell whether solutions exist, but thought to be very hard to tell how many. Many of these problems are #P-complete, and hence among the hardest problems in #P, since a polynomial time solution to any of them would allow a polynomial time solution to all other #P problems.

Problems in NP not known to be in P or NP-complete
In 1975, Richard E. Ladner showed that if P ≠ NP, then there exist problems in NP that are neither in P nor NP-complete. Such problems are called NP-intermediate problems. The graph isomorphism problem, the discrete logarithm problem, and the integer factorization problem are examples of problems believed to be NP-intermediate. They are some of the very few NP problems not known to be in P or to be NP-complete.

The graph isomorphism problem is the computational problem of determining whether two finite graphs are isomorphic. An important unsolved problem in complexity theory is whether the graph isomorphism problem is in P, NP-complete, or NP-intermediate. The answer is not known, but it is believed that the problem is at least not NP-complete. If graph isomorphism is NP-complete, the polynomial time hierarchy collapses to its second level. Since it is widely believed that the polynomial hierarchy does not collapse to any finite level, it is believed that graph isomorphism is not NP-complete. The best algorithm for this problem, due to László Babai, runs in quasi-polynomial time.

The integer factorization problem is the computational problem of determining the prime factorization of a given integer. Phrased as a decision problem, it is the problem of deciding whether the input has a factor less than k. No efficient integer factorization algorithm is known, and this fact forms the basis of several modern cryptographic systems, such as the RSA algorithm. The integer factorization problem is in NP and in co-NP (and even in UP and co-UP ). If the problem is NP-complete, the polynomial time hierarchy will collapse to its first level (i.e., NP = co-NP). The most efficient known algorithm for integer factorization is the general number field sieve, which takes expected time

 * $$O\left (\exp \left ( \left (\tfrac{64n}{9} \log(2) \right )^{\frac{1}{3}} \left ( \log(n\log(2)) \right )^{\frac{2}{3}} \right) \right )$$

to factor an n-bit integer. The best known quantum algorithm for this problem, Shor's algorithm, runs in polynomial time, although this does not indicate where the problem lies with respect to non-quantum complexity classes.

Does P mean "easy"?
All of the above discussion has assumed that P means "easy" and "not in P" means "difficult", an assumption known as Cobham's thesis. It is a common assumption in complexity theory; but there are caveats.

First, it can be false in practice. A theoretical polynomial algorithm may have extremely large constant factors or exponents, rendering it impractical. For example, the problem of deciding whether a graph G contains H as a minor, where H is fixed, can be solved in a running time of O(n2), where n is the number of vertices in G. However, the big O notation hides a constant that depends superexponentially on H. The constant is greater than $$ 2 \uparrow \uparrow (2 \uparrow \uparrow (2 \uparrow \uparrow (h/2) ) ) $$ (using Knuth's up-arrow notation), and where h is the number of vertices in H.

On the other hand, even if a problem is shown to be NP-complete, and even if P ≠ NP, there may still be effective approaches to the problem in practice. There are algorithms for many NP-complete problems, such as the knapsack problem, the traveling salesman problem, and the Boolean satisfiability problem, that can solve to optimality many real-world instances in reasonable time. The empirical average-case complexity (time vs. problem size) of such algorithms can be surprisingly low. An example is the simplex algorithm in linear programming, which works surprisingly well in practice; despite having exponential worst-case time complexity, it runs on par with the best known polynomial-time algorithms.

Finally, there are types of computations which do not conform to the Turing machine model on which P and NP are defined, such as quantum computation and randomized algorithms.

Reasons to believe P ≠ NP or P = NP
Cook provides a restatement of the problem in The P Versus NP Problem as "Does P = NP?" According to polls, most computer scientists believe that P ≠ NP. A key reason for this belief is that after decades of studying these problems no one has been able to find a polynomial-time algorithm for any of more than 3000 important known NP-complete problems (see List of NP-complete problems). These algorithms were sought long before the concept of NP-completeness was even defined (Karp's 21 NP-complete problems, among the first found, were all well-known existing problems at the time they were shown to be NP-complete). Furthermore, the result P = NP would imply many other startling results that are currently believed to be false, such as NP = co-NP and P = PH.

It is also intuitively argued that the existence of problems that are hard to solve but for which the solutions are easy to verify matches real-world experience. "Scott Aaronson, UT Austin"

On the other hand, some researchers believe that there is overconfidence in believing P ≠ NP and that researchers should explore proofs of P = NP as well. For example, in 2002 these statements were made: "The main argument in favor of P ≠ NP is the total lack of fundamental progress in the area of exhaustive search. This is, in my opinion, a very weak argument. The space of algorithms is very large and we are only at the beginning of its exploration. [...] The resolution of Fermat's Last Theorem also shows that very simple questions may be settled only by very deep theories." "Being attached to a speculation is not a good guide to research planning. One should always try both directions of every problem. Prejudice has caused famous mathematicians to fail to solve famous problems whose solution was opposite to their expectations, even though they had developed all the methods required."

- Anil Nerode, Cornell University

DLIN vs NLIN
When one substitutes "linear time on a multitape Turing machine" for "polynomial time" in the definitions of P and NP, one obtains the classes DLIN and NLIN. It is known that DLIN≠NLIN.

Consequences of solution
One of the reasons the problem attracts so much attention is the consequences of the possible answers. Either direction of resolution would advance theory enormously, and perhaps have huge practical consequences as well.

P = NP
A proof that P = NP could have stunning practical consequences if the proof leads to efficient methods for solving some of the important problems in NP. The potential consequences, both positive and negative, arise since various NP-complete problems are fundamental in many fields.

It is also very possible that a proof would not lead to practical algorithms for NP-complete problems. The formulation of the problem does not require that the bounding polynomial be small or even specifically known. A non-constructive proof might show a solution exists without specifying either an algorithm to obtain it or a specific bound. Even if the proof is constructive, showing an explicit bounding polynomial and algorithmic details, if the polynomial is not very low-order the algorithm might not be sufficiently efficient in practice. In this case the initial proof would be mainly of interest to theoreticians, but the knowledge that polynomial time solutions are possible would surely spur research into better (and possibly practical) methods to achieve them.

A solution showing P = NP could upend the field of cryptography, which relies on certain problems being difficult. A constructive and efficient solution to an NP-complete problem such as 3-SAT would break most existing cryptosystems including: These would need modification or replacement with information-theoretically secure solutions that do not assume P ≠ NP.
 * Existing implementations of public-key cryptography, a foundation for many modern security applications such as secure financial transactions over the Internet.
 * Symmetric ciphers such as AES or 3DES, used for the encryption of communications data.
 * Cryptographic hashing, which underlies blockchain cryptocurrencies such as Bitcoin, and is used to authenticate software updates. For these applications, finding a pre-image that hashes to a given value must be difficult, ideally taking exponential time. If P = NP, then this can take polynomial time, through reduction to SAT.

There are also enormous benefits that would follow from rendering tractable many currently mathematically intractable problems. For instance, many problems in operations research are NP-complete, such as types of integer programming and the travelling salesman problem. Efficient solutions to these problems would have enormous implications for logistics. Many other important problems, such as some problems in protein structure prediction, are also NP-complete; making these problems efficiently solvable could considerably advance life sciences and biotechnology.

These changes could be insignificant compared to the revolution efficiently solving NP-complete problems would cause in mathematics itself. Gödel, in his early thoughts on computational complexity, noted that a mechanical method that could solve any problem would revolutionize mathematics: "If there really were a machine with φ(n) ∼ k⋅n (or even ∼ k⋅n2), this would have consequences of the greatest importance. Namely, it would obviously mean that in spite of the undecidability of the Entscheidungsproblem, the mental work of a mathematician concerning Yes-or-No questions could be completely replaced by a machine. After all, one would simply have to choose the natural number n so large that when the machine does not deliver a result, it makes no sense to think more about the problem."

Similarly, Stephen Cook (assuming not only a proof, but a practically efficient algorithm) says: "... it would transform mathematics by allowing a computer to find a formal proof of any theorem which has a proof of a reasonable length, since formal proofs can easily be recognized in polynomial time. Example problems may well include all of the CMI prize problems."

Research mathematicians spend their careers trying to prove theorems, and some proofs have taken decades or even centuries to find after problems have been stated—for instance, Fermat's Last Theorem took over three centuries to prove. A method guaranteed to find a proof if a "reasonable" size proof exists, would essentially end this struggle.

Donald Knuth has stated that he has come to believe that P = NP, but is reserved about the impact of a possible proof: "[...] if you imagine a number M that's finite but incredibly large—like say the number 10↑↑↑↑3 discussed in my paper on "coping with finiteness"—then there's a humongous number of possible algorithms that do nM bitwise or addition or shift operations on n given bits, and it's really hard to believe that all of those algorithms fail.

My main point, however, is that I don't believe that the equality P = NP will turn out to be helpful even if it is proved, because such a proof will almost surely be nonconstructive."

P ≠ NP
A proof of P ≠ NP would lack the practical computational benefits of a proof that P = NP, but would represent a great advance in computational complexity theory and guide future research. It would demonstrate that many common problems cannot be solved efficiently, so that the attention of researchers can be focused on partial solutions or solutions to other problems. Due to widespread belief in P ≠ NP, much of this focusing of research has already taken place.

P ≠ NP still leaves open the average-case complexity of hard problems in NP. For example, it is possible that SAT requires exponential time in the worst case, but that almost all randomly selected instances of it are efficiently solvable. Russell Impagliazzo has described five hypothetical "worlds" that could result from different possible resolutions to the average-case complexity question. These range from "Algorithmica", where P = NP and problems like SAT can be solved efficiently in all instances, to "Cryptomania", where P ≠ NP and generating hard instances of problems outside P is easy, with three intermediate possibilities reflecting different possible distributions of difficulty over instances of NP-hard problems. The "world" where P ≠ NP but all problems in NP are tractable in the average case is called "Heuristica" in the paper. A Princeton University workshop in 2009 studied the status of the five worlds.

Results about difficulty of proof
Although the P = NP problem itself remains open despite a million-dollar prize and a huge amount of dedicated research, efforts to solve the problem have led to several new techniques. In particular, some of the most fruitful research related to the P = NP problem has been in showing that existing proof techniques are insufficient for answering the question, suggesting novel technical approaches are required.

As additional evidence for the difficulty of the problem, essentially all known proof techniques in computational complexity theory fall into one of the following classifications, all insufficient to prove P ≠ NP:

These barriers are another reason why NP-complete problems are useful: if a polynomial-time algorithm can be demonstrated for an NP-complete problem, this would solve the P = NP problem in a way not excluded by the above results.

These barriers lead some computer scientists to suggest the P versus NP problem may be independent of standard axiom systems like ZFC (cannot be proved or disproved within them). An independence result could imply that either P ≠ NP and this is unprovable in (e.g.) ZFC, or that P = NP but it is unprovable in ZFC that any polynomial-time algorithms are correct. However, if the problem is undecidable even with much weaker assumptions extending the Peano axioms for integer arithmetic, then nearly polynomial-time algorithms exist for all NP problems. Therefore, assuming (as most complexity theorists do) some NP problems don't have efficient algorithms, proofs of independence with those techniques are impossible. This also implies proving independence from PA or ZFC with current techniques is no easier than proving all NP problems have efficient algorithms.

Logical characterizations
The P = NP problem can be restated as certain classes of logical statements, as a result of work in descriptive complexity.

Consider all languages of finite structures with a fixed signature including a linear order relation. Then, all such languages in P are expressible in first-order logic with the addition of a suitable least fixed-point combinator. Recursive functions can be defined with this and the order relation. As long as the signature contains at least one predicate or function in addition to the distinguished order relation, so that the amount of space taken to store such finite structures is actually polynomial in the number of elements in the structure, this precisely characterizes P.

Similarly, NP is the set of languages expressible in existential second-order logic—that is, second-order logic restricted to exclude universal quantification over relations, functions, and subsets. The languages in the polynomial hierarchy, PH, correspond to all of second-order logic. Thus, the question "is P a proper subset of NP" can be reformulated as "is existential second-order logic able to describe languages (of finite linearly ordered structures with nontrivial signature) that first-order logic with least fixed point cannot?". The word "existential" can even be dropped from the previous characterization, since P = NP if and only if P = PH (as the former would establish that NP = co-NP, which in turn implies that NP = PH).

Polynomial-time algorithms
No known algorithm for a NP-complete problem runs in polynomial time. However, there are algorithms known for NP-complete problems that if P = NP, the algorithm runs in polynomial time on accepting instances (although with enormous constants, making the algorithm impractical). However, these algorithms do not qualify as polynomial time because their running time on rejecting instances are not polynomial. The following algorithm, due to Levin (without any citation), is such an example below. It correctly accepts the NP-complete language SUBSET-SUM. It runs in polynomial time on inputs that are in SUBSET-SUM if and only if P = NP:

// Algorithm that accepts the NP-complete language SUBSET-SUM. // // this is a polynomial-time algorithm if and only if P = NP. // // "Polynomial-time" means it returns "yes" in polynomial time when // the answer should be "yes", and runs forever when it is "no". // // Input: S = a finite set of integers // Output: "yes" if any subset of S adds up to 0. // Runs forever with no output otherwise. // Note: "Program number M" is the program obtained by // writing the integer M in binary, then // considering that string of bits to be a ''// program. Every possible program can be'' // generated this way, though most do nothing // because of syntax errors. FOR K = 1...∞ FOR M = 1...K    Run program number M for K steps with input S     IF the program outputs a list of distinct integers AND the integers are all in S      AND the integers sum to 0 THEN OUTPUT "yes" and HALT

This is a polynomial-time algorithm accepting an NP-complete language only if P = NP. "Accepting" means it gives "yes" answers in polynomial time, but is allowed to run forever when the answer is "no" (also known as a semi-algorithm).

This algorithm is enormously impractical, even if P = NP. If the shortest program that can solve SUBSET-SUM in polynomial time is b bits long, the above algorithm will try at least $2^{b} − 1$ other programs first.

P and NP
A decision problem is a problem that takes as input some string w over an alphabet Σ, and outputs "yes" or "no". If there is an algorithm (say a Turing machine, or a computer program with unbounded memory) that produces the correct answer for any input string of length n in at most cnk steps, where k and c are constants independent of the input string, then we say that the problem can be solved in polynomial time and we place it in the class P. Formally, P is the set of languages that can be decided by a deterministic polynomial-time Turing machine. Meaning,
 * $$\mathbf{P} = \{ L : L=L(M) \text{ for some deterministic polynomial-time Turing machine } M \}$$

where
 * $$L(M) = \{ w\in\Sigma^{*}: M \text{ accepts } w \}$$

and a deterministic polynomial-time Turing machine is a deterministic Turing machine M that satisfies two conditions:

 * 1) M halts on all inputs w and
 * 2) there exists $$k \in N$$ such that $$T_M(n)\in O(n^k)$$, where O refers to the big O notation and
 * $$T_M(n) = \max\{ t_M(w) : w\in\Sigma^{*}, |w| = n \}$$
 * $$t_M(w) = \text{ number of steps }M\text{ takes to halt on input }w.$$

NP can be defined similarly using nondeterministic Turing machines (the traditional way). However, a modern approach uses the concept of certificate and verifier. Formally, NP is the set of languages with a finite alphabet and verifier that runs in polynomial time. The following defines a "verifier":

Let L be a language over a finite alphabet, Σ.

L ∈ NP if, and only if, there exists a binary relation $$R\subset\Sigma^{*}\times\Sigma^{*}$$ and a positive integer k such that the following two conditions are satisfied:

 * 1) For all $$x\in\Sigma^{*}$$, $$x\in L \Leftrightarrow\exists y\in\Sigma^{*}$$ such that (x, y) ∈ R and $$|y|\in O(|x|^k)$$ ; and
 * 2) the language $$L_{R} = \{ x\# y:(x,y)\in R\}$$ over $$\Sigma\cup\{\#\}$$ is decidable by a deterministic Turing machine in polynomial time.

A Turing machine that decides LR is called a verifier for L and a y such that (x, y) ∈ R is called a certificate of membership of x in L.

Not all verifiers must be polynomial-time. However, for L to be in NP, there must be a verifier that runs in polynomial time.

Example
Let
 * $$\mathrm{COMPOSITE} = \left \{x\in\mathbb{N} \mid x=pq \text{ for integers } p, q > 1 \right \}$$
 * $$R = \left \{(x,y)\in\mathbb{N} \times\mathbb{N} \mid 1<y \leq \sqrt x \text{ and } y \text{ divides } x \right \}.$$

Whether a value of x is composite is equivalent to of whether x is a member of COMPOSITE. It can be shown that COMPOSITE ∈ NP by verifying that it satisfies the above definition (if we identify natural numbers with their binary representations).

COMPOSITE also happens to be in P, a fact demonstrated by the invention of the AKS primality test.

NP-completeness
There are many equivalent ways of describing NP-completeness.

Let L be a language over a finite alphabet Σ.

L is NP-complete if, and only if, the following two conditions are satisfied:

 * 1) L ∈ NP; and
 * 2) any L' in NP is polynomial-time-reducible to L (written as $$L' \leq_{p} L$$), where $$L' \leq_{p} L$$ if, and only if, the following two conditions are satisfied:
 * 3) There exists f : Σ* → Σ* such that for all w in Σ* we have: $$(w\in L' \Leftrightarrow f(w)\in L)$$; and
 * 4) there exists a polynomial-time Turing machine that halts with f(w) on its tape on any input w.

Alternatively, if L ∈ NP, and there is another NP-complete problem that can be polynomial-time reduced to L, then L is NP-complete. This is a common way of proving some new problem is NP-complete.

Claimed solutions
While the P versus NP problem is generally considered unsolved, many amateur and some professional researchers have claimed solutions. Gerhard J. Woeginger compiled a list of 116 purported proofs from 1986 to 2016, of which 61 were proofs of P = NP, 49 were proofs of P ≠ NP, and 6 proved other results, e.g. that the problem is undecidable. Some attempts at resolving P versus NP have received brief media attention, though these attempts have been refuted.

Popular culture
The film Travelling Salesman, by director Timothy Lanzone, is the story of four mathematicians hired by the US government to solve the P versus NP problem.

In the sixth episode of The Simpsons' seventh season "Treehouse of Horror VI", the equation P = NP is seen shortly after Homer accidentally stumbles into the "third dimension".

In the second episode of season 2 of Elementary, "Solve for X" Sherlock and Watson investigate the murders of mathematicians who were attempting to solve P versus NP.

Similar problems

 * A similar problem exists in the theory of algebraic complexity: VP vs. VNP problem. This problem has not been solved yet.

license: Creative Commons - Attribution Share-Alike - https://creativecommons.org/licenses/by-sa/4.0/

title: P versus NP problem