The exponential function (exp)

Rufus Fraanje

The exponential function \(e^x\), sometimes written as \(\mathrm{exp}(x)\), is a famous function with many applications, far more than just exponential growth and decay. It is defined by the power series 1 2,

\begin{equation*} \begin{array}{lcl} \displaystyle e^x &=& \displaystyle \sum_{n=0}^\infty \frac{x^n}{n!} \\ &=& \displaystyle 1 + x + \frac{x^2}{2} + \frac{x^3}{6} + \frac{x^4}{24} + \frac{x^5}{120}+\cdots \end{array} \end{equation*}

with \(n!\) the faculty of the integer \(n\) and the number \(e\,=\,\sum_{n=0}^\infty \frac{1}{n!}\,\approx\,2.718....\) the constant of Euler, after the Swiss mathematician Leonhard Euler (1707 - 1783) 3.

In some sense, the exponential function is a generalization of the sine and the cosine functions, \(\mathrm{sin}(x)\) and \(\mathrm{cos}(x)\) respectively, which are defined as 4

\begin{equation*} \begin{array}{lclcl} \mathrm{sin}(x)&=&\displaystyle\sum_{n=0}^\infty \frac{(-1)^nx^{2n+1}}{(2n+1)!}&=&\displaystyle x-\frac{x^3}{6}+\frac{x^5}{120}-\cdots \\ \mathrm{cos}(x)&=&\displaystyle\sum_{n=0}^\infty \frac{(-1)^nx^{2n}}{(2n)!}&=&\displaystyle 1-\frac{x^2}{2}+\frac{x^4}{24}-\cdots \end{array} \end{equation*}

One may recognize the similarities in the coefficients with the power series of the \(\mathrm{exp}(x)\) function. More precisely, when we evaluate the \(\mathrm{exp}(x)\) function for pure imaginary values of \(x\), that is \(x=i\alpha\) with \(\alpha\in\mathbb{R}\) an arbitrary real number and \(i\) the imaginary unit 5, such that \(i^2=-1\), then one can write

\begin{equation*} e^{i\alpha}~=~\cos(\alpha) + i\sin(\alpha) \end{equation*}

which is known as the Euler's formula 6. So, by applying the exponential function to imaginary numbers the exponential function splits in some sense in two separate functions: the real part being the cosine and the imaginary part being the sine function. This 'splitting' is thanks to the property of the imaginary unit \(i\) that by squaring \(i\) it yields a real number (\(i^2\,=\,-1\)), and thus \(i^n\) is real for even and imaginary for odd values of \(n\) in the power series above. Moreover, the alternating signs in the power series of the sine and the cosine functions are due to the fact that \((-1)^n\) is \(+1\) for even and \(-1\) for odd values of \(n\).

Thanks to this splitting in sine and cosine functions, the exponential function applied to imaginary or complex numbers can be used to describe oscillatory behavior in dynamic systems, as well as making rotations, projections and reflections in geometry. To be able to perform this in three rather than two dimensions, the imaginary or complex number may be replaced by a so called quaternion 7 or an even more general rotor or motor in the field of geometric algebra 8, which may be the topic of another blog post.

Evaluating Euler's formula at \(\alpha=\pi\), with \(\pi\) the circle constant 9, yields the Euler's identity 10

\begin{equation*} e^{i\pi}+1~=~0 \end{equation*}

that beautifully relates five mathematical constants: \(0,\,1,\,i,\,\pi\) and \(e\).

The inverse of the exponential function is the natural logarithm function \(\mathrm{ln}(x)\), such that \(\mathrm{ln}(e^x)\,=\,x\) and in addition, provided \(x>0\), \(e^{\mathrm{ln}(x)}\,=\,x\) 11 12. The general \(x^{\mathrm{th}}\) power of some value \(a\), i.e. \(a^x\) can be written in terms of the exponential function 13

\begin{equation*} a^x~=~e^{x \mathrm{ln}(a)}. \end{equation*}

This gives a way to compute \(a^x\) for non-integer values of \(x\) (\(a,x\) may even be complex valued).

Graph of the real and complex part of the exponential function evaluated on complex values.

Graphs of the real and imaginary part of the exponential function evaluated on complex values \(\alpha+i\beta\), generated by this Python script.

Solving differential equations

The exponential function appears as a solution or part of a solution of an equation with differentials, better known as a differential equation 14, e.g. of the form

\begin{equation*} \frac{\mathrm{d} y}{\mathrm{d} x}~=~ a y(x). \end{equation*}

Here \(x\) may represent a position in a spatial dimension, e.g. in meters, and the function \(y(x)\) may represent the spatial distribution of a physical quantity, e.g. units of strain or units of electrical charge. Differential equations also show up in describing dynamical systems, then \(x\) represents time (usually represented by the symbol \(t\) rather than \(x\)), expressed e.g. in units of seconds, and \(y(x)\) represents the position or velocity of a particle as a function of time. Also note, that the (physical) unit of \(a\) is the inverse of the unit of \(x\), to have proper units on both sides of the differential equation, i.e. units of \(y\) per units of \(x\). Hence, if \(x\) is measured in \(\mathrm{s}\) (seconds), than \(a\) has the unit \(1/\mathrm{s}\,=\,\mathrm{s}^{-1}\).

The exponential functions appears in the solution \(y(x)\) of the differential equation, because the exponential function is equal to its derivative, i.e.

\begin{equation*} \frac{\mathrm{d}e^x}{\mathrm{d} x}~=~ e^x. \end{equation*}

This property can be verified by term for term differentiating the power series expansion of the exponential function given in the first equation of this article. Therefore \(e^x\) satisfies the differential equation

\begin{equation*} \frac{\mathrm{d} y}{\mathrm{d} x}~=~ y(x). \end{equation*}

But, note that there are more functions satisfying this differential equation, all are of the form \(y(x)\,=\,ce^x\) with \(c\) some number. This number can be determined uniquely if in addition to the differential equation we add another equation, also known as a constraint, initial condition or boundary condition. For example, when \(y(0)=3\) it follows that \(c=3\), because \(e^0\,=\,1\). Also note, that the (physical) unit of \(c\) is the same as the unit of \(y\).

Thanks to the chain rule in calculus 15 it can be shown that

\begin{equation*} \frac{\mathrm{d}e^{ax}}{\mathrm{d}x} ~=~ ae^{ax} \end{equation*}

which gives a solution to the differential equation depending on \(a\) given above, and all solutions are of the form \(y(x)\,=\,ce^{ax}\), with \(c\) some number determined by the additional constraint. The parameter \(a\) of the differential equation, can be seen as a scaling parameter of the spatial or temporal dimension \(x\). This can be seen by dividing the differential equation by \(a\), resulting in

\begin{equation*} \frac{\mathrm{d} y}{a\mathrm{d} x}~=~ y(x) \end{equation*}

which, by introducing the scaled dimension parameter \(x'\,=\,ax\), can be rewritten in the normalized differential equation

\begin{equation*} \frac{\mathrm{d} y}{\mathrm{d} x'}~=~ y(x'). \end{equation*}

Solutions to this differential equation has been shown above as having the form \(y(x')=ce^{x'}\), which is only valid over the scaled (or normalized) dimension parameter \(x'\,=\,ax\). Rewriting in terms of the unscaled dimension parameter \(x\) yields the form we obtained by applying the chain rule

\begin{equation*} y(x)~=~ce^{ax}. \end{equation*}

As we have seen above, the parameters \(c\) and \(a\) are both depending on the units that have been chosen for \(y\) and \(x\) respectively. In fact these parameters determine how much we have to scale down \(y\) by \(c\) and stretch \(x\) by \(a\) to arrive at the exponential function \(e^{x'}\). Or stated otherwise, \(c\) determines how much units the physical quantity \(y\) represents and the parameter \(a\) specifies how fast \(y\) is changing as \(x\) evolves, which is after all just what the differential equation \(\mathrm{d}y/\mathrm{d}x\,=\,ay(x)\) is saying.

Besides the special case of \(a=1\) leading to the normalized differential equation, it is illustrative to interpret the differential equation and its solution for \(a=0\), \(a=-1\), \(a\to+\infty\) and \(a\to-\infty\). Note, that nothing withholds you (and is really rewarding) to study the equations for imaginary values \(a=i\alpha\) (\(\alpha\in\mathbb{R}\)) and more general complex values \(a=\alpha+i\beta\) (\(\alpha,\beta\in\mathbb{R}\)) as well (c.f. the graph above)!

The matrix exponential

The above discussion has been performed for \(x\) and \(y\) being scalar quantities and the exponential function being a scalar function. However, the exponential function is not limited to scalars, it can be applied on other mathematical objects such as a matrix as well. Therefore, let \(X\) be a square matrix, then the matrix exponential function is defined (similarly as above in the scalar case) by 16

\begin{equation*} e^X~=~\sum_{n=0}^\infty \frac{X^n}{n!}~=~I + X + \frac{X^2}{2} + \frac{X^3}{6} + \frac{x^4}{24} + \frac{X^5}{120} + \cdots. \end{equation*}

where \(I\) the identity matrix having the same dimension as \(X\). So the matrix exponential \(e^X\) is a square matrix as well having the same dimension as \(X\).

In the following we replace \(X\) by the quantity \(At\) where \(A\) a square matrix, which generalizes the parameter \(a\) above to the matrix case, and \(t\) a scalar quantity representing e.g. time or displacement in a fixed direction, similar as the parameter \(x\) above. Then, the matrix exponential function \(e^{At}\) has a beautiful derivative with respect to the scalar quantity \(t\), given by

\begin{equation*} \frac{\mathrm{d}e^{At}}{\mathrm{d}t}~=~Ae^{At}, \end{equation*}

which can be verified, similar as in the scalar case, by taking term for term the derivative of the power series of the matrix exponential function. Note, we could have written \(\mathrm{d}e^{At}/\mathrm{d}t\,=\,e^{At}A\) as well, so \(A\) commutes with \(e^{At}\), though matrix products do not commute in general!

This result can be applied to solve coupled differential equations, that show up in e.g. linear dynamic systems with multiple lumped components, such as mass-spring-damper systems or RLC-networks in the electrical domain. Often, the differential equations can be ordered in such a way that they can be written in the form of the following vector differential equation

\begin{equation*} \frac{\mathrm{d}y}{\mathrm{d}t}~=~ A y(t) \end{equation*}

where now \(y(t)\) is a (column) vector signal having as many elements as the rows or columns of the square matrix \(A\). Note, that to have the units being identical on both sides of this vector differential equation, the unit of \(A\) should be the inverse of the unit of \(t\). For dynamic systems \(t\) usually has the unit \(\mathrm{s}\) and thus \(A\) has the unit \(\mathrm{s}^{-1}\).

The solutions to the vector differential equation have the form

\begin{equation*} y(t)~=~e^{At}c \end{equation*}

where \(c\) a (column) vector having the same dimension and unit as \(y\), and can be determined when an additional condition on \(y(t)\) for some \(t\) is added, often an initial condition \(y(0)=c\) because \(e^{At}\,=\,I\) for \(t=0\).

For the scalar case when \(A\,=\,a\in\mathbb{R}\) or \(\mathbb{C}\) the evolution of \(e^{at}\) is as studied above. The case \(A\) is a \(2\times2\) matrix or of higher dimension is more complex. However, the scalar case simply extends to the matrix case when \(A\) is a diagonal matrix, i.e. a matrix with arbitrary values \(a_k\) on the diagonal and zeros elsewhere. Here, the subscript \(k\) is an integer referring to the corresponding position on the diagonal of \(A\). In this case of diagonal \(A\), the matrix exponential \(e^{At}\) is a diagonal matrix as well with diagonal elements \(e^{a_kt}\), similar to the scalar case. In fact, when \(A\) is diagonal the equations in the vector differential equation are not coupled and can be considered as multiple scalar differential equations.

More difficult is the situation when one or more off-diagonal elements of \(A\) are nonzero. In this case the matrix should be made diagonal or 'almost' diagonal by making certain transformations, based on the eigenvalue decomposition 17 or the Jordan normal form 18 respectively. The resulting solutions contain \(e^{\lambda_k t}\) where \(\lambda_k\) an eigenvalue that has the same meaning as \(a\) in the scalar case, and can be:

real, equivalent to exponential growth or decay;
imaginary, harmonic oscillatory behavior;
complex, growing or decaying harmonic oscillatory behavior.

In another blog post, we may go in further detail about this, for now we refer to some YouTube videos of MIT math professor Gilbert Strang on Eigenvalues and Eigenvectors 19 and The Matrix Exponential 20.

In this article we have only considered autonomous systems, i.e. systems without applying external inputs, such as disturbances or controls. Extending to dynamic systems with external inputs will result in the so called State-space representation 21, that may also be the topic of another post.

I would like to finish this article on the exponential function with pointing to another inspiring YouTube video by Colin Smith on Physics in Clojure 22, that ends with a weird trick 23 obtained by applying the exponential function not on a number but on the differential operator, which however is very similar to the Laplace and Fourier transforms, that are on their turn strongly build on the exponential function as well.

I hope you found this article informative. Please, let me known when you liked it or have some questions or comments.

1: NIST Digital Library of Mathematical Functions: §4.2(iii) The Exponential Function
2: Wikipedia: Exponential function
3: Wikipedia: Leonhard Euler
4: Wikipedia: Taylor series
5: Wikipedia: Imaginary unit
6: Wikipedia: Euler's formula
7: Wikipedia: Quaternion
8: Wikipedia: Geometric algebra
9: Wikipedia: Pi
10: Wikipedia: Euler's identity
11: Wikipedia: Natural logarithm
12: NIST Digital Library of Mathematical Functions: §4.2(i) The Logarithm
13: NIST Digital Library of Mathematical Functions: §4.2(iv) Powers
14: Wikipedia: Differential equation
15: Wikipedia: Chain rule
16: Wikipedia: Matrix exponential
17: Wikipedia: Eigenvalues and eigenvectors
18: Wikipedia: Jordan normal form
19: Gilbert Strang, Eigenvalues and Eigenvectors (YouTube MIT OpenCourseWare)
20: Gilbert Strang, The Matrix Exponential (YouTube MIT OpenCourseWare)
21: Wikipedia: State-space representation
22: Colin Smith, Physics in Clojure (YouTube Clojure/West)
23: Colin Smith, One weird trick (in: Physics in Clojure at 38:15)

Solving differential equations

The matrix exponential

Comments