outer product

Given two vectors of size $m\; \backslash times\; 1$ and $n\; \backslash times\; 1$ respectively

:$$\backslash mathbf\{u\}\; =\; \backslash begin\{bmatrix\}\; u\_1\; \backslash \backslash \; u\_2\; \backslash \backslash \; \backslash vdots\; \backslash \backslash \; u\_m\; \backslash end\{bmatrix\},$$

\quad

\mathbf{v} = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}

their outer product, denoted $\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\},$ is defined as the $m\; \backslash times\; n$ matrix $\backslash mathbf\{A\}$ obtained by multiplying each element of $\backslash mathbf\{u\}$ by each element of {{nowrap|$\backslash mathbf\{v\}$:}}{{cite book |title=Encyclopaedia of Physics |edition=2nd |first1=R. G. |last1=Lerner |author1-link=Rita G. Lerner |first2=G. L. |last2=Trigg |publisher=VHC |year=1991 |isbn=0-89573-752-3 |url-access=registration |url=https://archive.org/details/encyclopediaofph00lern}}

:$$$$

\mathbf{u} \otimes \mathbf{v} = \mathbf{A} =

\begin{bmatrix}

u_1v_1 & u_1v_2 & \dots & u_1v_n \\

u_2v_1 & u_2v_2 & \dots & u_2v_n \\

\vdots & \vdots & \ddots & \vdots \\

u_mv_1 & u_mv_2 & \dots & u_mv_n

\end{bmatrix}

Or, in index notation:

:$$(\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\})\_\{ij\}\; =\; u\_i\; v\_j$$

Denoting the dot product by $\backslash ,\backslash cdot,\backslash ,$ if given an $n\; \backslash times\; 1$ vector $\backslash mathbf\{w\},$ then $(\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\})\; \backslash mathbf\{w\}\; =\; (\backslash mathbf\{v\}\; \backslash cdot\; \backslash mathbf\{w\})\; \backslash mathbf\{u\}.$ If given a $1\; \backslash times\; m$ vector $\backslash mathbf\{x\},$ then $\backslash mathbf\{x\}\; (\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\})\; =\; (\backslash mathbf\{x\}\; \backslash cdot\; \backslash mathbf\{u\})\; \backslash mathbf\{v\}^\{\backslash operatorname\{T\}\}.$

If $\backslash mathbf\{u\}$ and $\backslash mathbf\{v\}$ are vectors of the same dimension bigger than 1, then $\backslash det\; (\backslash mathbf\{u\}\; \backslash otimes\backslash mathbf\{v\})\; =\; 0$.

The outer product $\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\}$ is equivalent to a matrix multiplication $\backslash mathbf\{u\}\; \backslash mathbf\{v\}^\{\backslash operatorname\{T\}\},$ provided that $\backslash mathbf\{u\}$ is represented as a $m\; \backslash times\; 1$ column vector and $\backslash mathbf\{v\}$ as a $n\; \backslash times\; 1$ column vector (which makes $\backslash mathbf\{v\}^\{\backslash operatorname\{T\}\}$ a row vector).{{cite book |title=Linear Algebra |edition=4th |first1=S. |last1=Lipschutz |first2=M. |last2=Lipson |series=Schaum’s Outlines |publisher=McGraw-Hill |year=2009 |isbn=978-0-07-154352-1}}{{cite web |last=Keller |first=Frank |date=February 23, 2020 |title=Algebraic Properties of Matrices; Transpose; Inner and Outer Product |url=https://www.inf.ed.ac.uk/teaching/courses/cfcs1/lectures/cfcs_l10.pdf |archive-url=https://web.archive.org/web/20171215061654/http://www.inf.ed.ac.uk/teaching/courses/cfcs1/lectures/cfcs_l10.pdf |archive-date=2017-12-15 |url-status=live |access-date=September 6, 2020 |website=inf.ed.ac.uk}} For instance, if $m\; =\; 4$ and $n\; =\; 3,$ thenJames M. Ortega (1987) Matrix Theory: A Second Course, page 7, Plenum Press {{ISBN|0-306-42433-9}}

:$$$$

\mathbf{u} \otimes \mathbf{v} = \mathbf{u}\mathbf{v}^\textsf{T} =

\begin{bmatrix}u_1 \\ u_2 \\ u_3 \\ u_4\end{bmatrix}

\begin{bmatrix}v_1 & v_2 & v_3\end{bmatrix} =

\begin{bmatrix}

u_1 v_1 & u_1 v_2 & u_1 v_3 \\

u_2 v_1 & u_2 v_2 & u_2 v_3 \\

u_3 v_1 & u_3 v_2 & u_3 v_3 \\

u_4 v_1 & u_4 v_2 & u_4 v_3

\end{bmatrix}.

For complex vectors, it is often useful to take the conjugate transpose of $\backslash mathbf\{v\},$ denoted $\backslash mathbf\{v\}^\backslash dagger$ or $\backslash left(\backslash mathbf\{v\}^\backslash textsf\{T\}\backslash right)^*$:

:$$\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\}\; =\; \backslash mathbf\{u\}\; \backslash mathbf\{v\}^\backslash dagger\; =\; \backslash mathbf\{u\}\; \backslash left(\backslash mathbf\{v\}^\backslash textsf\{T\}\backslash right)^*.$$

If $m\; =\; n,$ then one can take the matrix product the other way, yielding a scalar (or $1\; \backslash times\; 1$ matrix):

:$$\backslash left\backslash langle\backslash mathbf\{u\},\; \backslash mathbf\{v\}\backslash right\backslash rangle\; =\; \backslash mathbf\{u\}^\backslash textsf\{T\}\; \backslash mathbf\{v\}$$

which is the standard inner product for Euclidean vector spaces, better known as the dot product. The dot product is the trace of the outer product.{{cite book |first=Robert F. |last=Stengel |title=Optimal Control and Estimation |location=New York |publisher=Dover Publications |year=1994 |page=26 |isbn=0-486-68200-5 |url=https://books.google.com/books?id=jDjPxqm7Lw0C&pg=PA26}} Unlike the dot product, the outer product is not commutative.

Multiplication of a vector $\backslash mathbf\{w\}$ by the matrix $\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\}$ can be written in terms of the inner product, using the relation $\backslash left(\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\}\backslash right)\backslash mathbf\{w\}\; =\; \backslash mathbf\{u\}\backslash left\backslash langle\backslash mathbf\{v\},\; \backslash mathbf\{w\}\backslash right\backslash rangle$.

Given two tensors $\backslash mathbf\{u\},\; \backslash mathbf\{v\}$ with dimensions $(k\_1,\; k\_2,\; \backslash dots,\; k\_m)$ and $(l\_1,\; l\_2,\; \backslash dots,\; l\_n)$, their outer product $\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\}$ is a tensor with dimensions $(k\_1,\; k\_2,\; \backslash dots,\; k\_m,\; l\_1,\; l\_2,\; \backslash dots,\; l\_n)$ and entries

:$$(\backslash mathbf\{u\}\; \backslash otimes\; \backslash mathbf\{v\})\_\{i\_1,\; i\_2,\; \backslash dots\; i\_m,\; j\_1,\; j\_2,\; \backslash dots,\; j\_n\}\; =\; u\_\{i\_1,\; i\_2,\; \backslash dots,\; i\_m\}\; v\_\{j\_1,\; j\_2,\; \backslash dots,\; j\_n\}$$

For example, if $\backslash mathbf\{A\}$ is of order 3 with dimensions $(3,\; 5,\; 7)$ and $\backslash mathbf\{B\}$ is of order 2 with dimensions $(10,\; 100),$ then their outer product $\backslash mathbf\{C\}$ is of order 5 with dimensions $(3,\; 5,\; 7,\; 10,\; 100).$ If $\backslash mathbf\{A\}$ has a component {{math|1=A_{[2, 2, 4]} = 11}} and $\backslash mathbf\{B\}$ has a component {{math|1=B_{[8, 88]} = 13}}, then the component of $\backslash mathbf\{C\}$ formed by the outer product is {{math|1=C_{[2, 2, 4, 8, 88]} = 143}}.

The outer product and Kronecker product are closely related; in fact the same symbol is commonly used to denote both operations.

If and , we have:

:$$\backslash begin\{align\}$$

\mathbf{u} \otimes_\text{Kron} \mathbf{v} &= \begin{bmatrix} 4 \\ 5 \\ 8 \\ 10 \\ 12 \\ 15\end{bmatrix}, &

\mathbf{u} \otimes_\text{outer} \mathbf{v} &= \begin{bmatrix} 4 & 5 \\ 8 & 10 \\ 12 & 15\end{bmatrix}

\end{align}

In the case of column vectors, the Kronecker product can be viewed as a form of vectorization (or flattening) of the outer product. In particular, for two column vectors $\backslash mathbf\{u\}$ and $\backslash mathbf\{v\}$, we can write:

:$$\backslash mathbf\{u\}\; \backslash otimes\_\{\backslash text\{Kron\}\}\; \backslash mathbf\{v\}\; =\; \backslash operatorname\{vec\}(\backslash mathbf\{v\}\; \backslash otimes\_\backslash text\{outer\}\; \backslash mathbf\{u\})$$

(The order of the vectors is reversed on the right side of the equation.)

Another similar identity that further highlights the similarity between the operations is

:$$\backslash mathbf\{u\}\; \backslash otimes\_\{\backslash text\{Kron\}\}\; \backslash mathbf\{v\}^\backslash textsf\{T\}\; =\; \backslash mathbf\; u\; \backslash mathbf\{v\}^\backslash textsf\{T\}\; =\; \backslash mathbf\{u\}\; \backslash otimes\_\{\backslash text\{outer\}\}\; \backslash mathbf\{v\}$$

where the order of vectors needs not be flipped. The middle expression uses matrix multiplication, where the vectors are considered as column/row matrices.

Given a pair of matrices $\backslash mathbf\{A\}$ of size $m\backslash times\; p$ and $\backslash mathbf\{B\}$ of size $p\backslash times\; n$, consider the matrix product $\backslash mathbf\{C\}\; =\; \backslash mathbf\{A\}\backslash ,\backslash mathbf\{B\}$ defined as usual as a matrix of size $m\backslash times\; n$.

Now let $\backslash mathbf\; a^\backslash text\{col\}\_k$ be the $k$-th column vector of $\backslash mathbf\; A$ and let $\backslash mathbf\; b^\backslash text\{row\}\_k$ be the $k$-th row vector of $\backslash mathbf\; B$. Then $\backslash mathbf\{C\}$ can be expressed as a sum of column-by-row outer products:

:$$\backslash mathbf\{C\}\; =\; \backslash mathbf\{A\}\backslash ,\; \backslash mathbf\{B\}\; =$$

\left(

\sum_{k=1}^p {A}_{ik}\, {B}_{kj}

\right)_{

\begin{matrix} 1\le i \le m \\[-20pt] 1 \le j\le n \end{matrix}

} =

\begin{bmatrix} & & \\ \mathbf a^\text{col}_{1} & \cdots & \mathbf a^\text{col}_{p} \\ & & \end{bmatrix}

\begin{bmatrix} & \mathbf b^\text{row}_{1} & \\ & \vdots & \\ & \mathbf b^\text{row}_{p} & \end{bmatrix}

= \sum_{k=1}^p \mathbf a^\text{col}_k \mathbf b^\text{row}_k

This expression has duality with the more common one as a matrix built with row-by-column inner product entries (or dot product): $C\_\{ij\}\; =\; \backslash langle\{\backslash mathbf\; a^\backslash text\{row\}\_i,\backslash ,\backslash mathbf\; b\_j^\backslash text\{col\}\}\backslash rangle$

This relation is relevant{{cite book |last2=Bau III |first2=David |last1=Trefethen |first1=Lloyd N. |author1-link=Lloyd N. Trefethen |title=Numerical linear algebra |publisher=Society for Industrial and Applied Mathematics |location=Philadelphia |isbn=978-0-89871-361-9 |year=1997}} in the application of the Singular Value Decomposition (SVD) (and Spectral Decomposition as a special case). In particular, the decomposition can be interpreted as the sum of outer products of each left ($\backslash mathbf\{u\}\_k$) and right ($\backslash mathbf\{v\}\_k$) singular vectors, scaled by the corresponding nonzero singular value $\backslash sigma\_k$:

:$$\backslash mathbf\{A\}\; =\; \backslash mathbf\{U\; \backslash Sigma\; V^T\}\; =\; \backslash sum\_\{k=1\}^\{\backslash operatorname\{rank\}(A)\}(\backslash mathbf\{u\}\_k\; \backslash otimes\; \backslash mathbf\{v\}\_k)\; \backslash ,\; \backslash sigma\_k$$

This result implies that $\backslash mathbf\{A\}$ can be expressed as a sum of rank-1 matrices with spectral norm $\backslash sigma\_k$ in decreasing order. This explains the fact why, in general, the last terms contribute less, which motivates the use of the truncated SVD as an approximation. The first term is the least squares fit of a matrix to an outer product of vectors.

The outer product of vectors satisfies the following properties:

:$$\backslash begin\{align\}$$

(\mathbf{u} \otimes \mathbf{v})^\textsf{T} &= (\mathbf{v} \otimes \mathbf{u}) \\

(\mathbf{v} + \mathbf{w}) \otimes \mathbf{u} &= \mathbf{v} \otimes \mathbf{u} + \mathbf{w} \otimes \mathbf{u} \\

\mathbf{u} \otimes (\mathbf{v} + \mathbf{w}) &= \mathbf{u} \otimes \mathbf{v} + \mathbf{u} \otimes \mathbf{w} \\

c (\mathbf{v} \otimes \mathbf{u}) &= (c\mathbf{v}) \otimes \mathbf{u} = \mathbf{v} \otimes (c\mathbf{u})

\end{align}

The outer product of tensors satisfies the additional associativity property:

:$$$$

(\mathbf{u} \otimes \mathbf{v}) \otimes \mathbf{w} =

\mathbf{u} \otimes (\mathbf{v} \otimes \mathbf{w})

If u and v are both nonzero, then the outer product matrix uv^T always has matrix rank 1. Indeed, the columns of the outer product are all proportional to u. Thus they are all linearly dependent on that one column, hence the matrix is of rank one.

("Matrix rank" should not be confused with "tensor order", or "tensor degree", which is sometimes referred to as "rank".)

Let {{mvar|V}} and {{mvar|W}} be two vector spaces. The outer product of $\backslash mathbf\; v\; \backslash in\; V$ and $\backslash mathbf\; w\; \backslash in\; W$ is the element $\backslash mathbf\; v\; \backslash otimes\; \backslash mathbf\; w\; \backslash in\; V\; \backslash otimes\; W$.

If {{mvar|V}} is an inner product space, then it is possible to define the outer product as a linear map {{math|V → W}}. In this case, the linear map $\backslash mathbf\; x\; \backslash mapsto\; \backslash langle\; \backslash mathbf\; v,\; \backslash mathbf\; x\backslash rangle$ is an element of the dual space of {{mvar|V}}, as this maps linearly a vector into its underlying field, of which $\backslash langle\; \backslash mathbf\; v,\; \backslash mathbf\; x\backslash rangle$ is an element. The outer product {{math|V → W}} is then given by

:$$(\backslash mathbf\; w\; \backslash otimes\; \backslash mathbf\; v)\; (\backslash mathbf\; x)\; =\; \backslash left\backslash langle\; \backslash mathbf\; v,\; \backslash mathbf\; x\; \backslash right\backslash rangle\; \backslash mathbf\; w.$$

This shows why a conjugate transpose of {{math|v}} is commonly taken in the complex case.

In some programming languages, given a two-argument function f (or a binary operator), the outer product, f, of two one-dimensional arrays, A and B, is a two-dimensional array C such that C[i, j] = f(A[i], B[j]). This is syntactically represented in various ways: in APL, as the infix binary operator ∘.f; in J, as the postfix adverb f/; in R, as the function outer(A, B, f) or the special %o%;{{cite web |title=outer function {{!}} R Documentation |url=https://www.rdocumentation.org/packages/base/versions/3.6.2/topics/outer|access-date=2020-09-07 |website=rdocumentation.org}} in Mathematica, as Outer[f, A, B]. In MATLAB, the function kron(A, B) is used for this product. These often generalize to multi-dimensional arguments, and more than two arguments.

In the Python library NumPy, the outer product can be computed with function np.outer().{{cite web |title=numpy.outer — NumPy v1.19 Manual |url=https://numpy.org/doc/stable/reference/generated/numpy.outer.html |access-date=2020-09-07 |website=numpy.org}} In contrast, np.kron results in a flat array. The outer product of multidimensional arrays can be computed using np.multiply.outer.

As the outer product is closely related to the Kronecker product, some of the applications of the Kronecker product use outer products. These applications are found in quantum theory, signal processing, and image compression.{{cite book |last1=Steeb |first1=Willi-Hans |title=Matrix Calculus and Kronecker Product: A Practical Approach to Linear and Multilinear Algebra |last2=Hardy |first2=Yorick |publisher=World Scientific |year=2011 |isbn=978-981-4335-31-7 |edition=2 |chapter=Applications (Chapter 3)}}

Suppose {{math|s, t, w, z ∈ C}} so that {{math|(s, t)}} and {{math|(w, z)}} are in {{math|C²}}. Then the outer product of these complex 2-vectors is an element of {{math|M(2, C)}}, the 2 × 2 complex matrices:

:

The determinant of this matrix is {{math|1=swtz − sztw = 0}} because of the commutative property of {{math|C}}.

In the theory of spinors in three dimensions, these matrices are associated with isotropic vectors due to this null property. Élie Cartan described this construction in 1937,Élie Cartan (1937) Lecons sur la theorie des spineurs, translated 1966: The Theory of Spinors, Hermann, Paris but it was introduced by Wolfgang Pauli in 1927Pertti Lounesto (1997) Clifford Algebras and Spinors, page 51, Cambridge University Press {{ISBN|0-521-59916-4}} so that {{math|M(2,C)}} has come to be called Pauli algebra.

The block form of outer products is useful in classification. Concept analysis is a study that depends on certain outer products:

When a vector has only zeros and ones as entries, it is called a logical vector, a special case of a logical matrix. The logical operation and takes the place of multiplication. The outer product of two logical vectors {{math|(u_i)}} and {{math|(v_j)}} is given by the logical matrix $\backslash left(a\_\{ij\}\backslash right)\; =\; \backslash left(u\_i\; \backslash land\; v\_j\backslash right)$. This type of matrix is used in the study of binary relations, and is called a rectangular relation or a cross-vector.Ki-Hang Kim (1982) Boolean Matrix Theory and Applications, page 37, Marcel Dekker {{ISBN|0-8247-1788-0}}

• Dyadics
• Householder transformation
• Norm (mathematics)
• Ricci calculus
• Scatter matrix

• Cartesian product
• Cross product
• Exterior product
• Hadamard product (matrices)

• Bra–ket notation for outer product
• Complex conjugate
• Conjugate transpose
• Transpose

{{Reflist}}

• {{cite book |last1=Carlen |first1=Eric |last2=Canceicao Carvalho |first2=Maria |title=Linear Algebra: From the Beginning |chapter=Outer Products and Orthogonal Projections |publisher=Macmillan |year=2006 |pages=217–218 |isbn=9780716748946 |chapter-url=https://books.google.com/books?id=EqGjBOXbPAoC&pg=PA217}}

{{Linear algebra}}

Category:Bilinear maps

Category:Higher-order functions

Category:Operations on vectors