Theorem – Cauchy-Schwarz Inequality

Theorem

Let $\mathbf{x}$ and $\mathbf{y}$ be vectors. Then,

$|\mathbf{x}\cdot\mathbf{y}| \leq |\mathbf{x}| |\mathbf{y}|$

I.e, the absolute value of the dot product of two vectors is less than or equal to the norm of $\mathbf{x}$ multiplied by the norm of $\mathbf{y}$ .

Proof

We split this proof into three cases. The first case is that $\mathbf{y} = \mathbf{0}$ and $\mathbf{x}$ are linearly dependent vectors. Since $\mathbf{y}$ is the zero vector, we have that $|\mathbf{x\cdot{y}}| = |0| = 0$ , and we have that the norm of $\mathbf{y}$ is also equal to zero. Thus, the equality becomes

$\begin{aligned} |0| &\leq |\mathbf{x}| |0| \\\\ \implies &0\leq{0} \end{aligned}$

Which is true. The next case is that $\mathbf{y} \ne \mathbf{0}$ and $\mathbf{x}$ are linearly dependent vectors. It follows that $\mathbf{x} = k\mathbf{y}$ (proof given here) since the vectors are linearly depenedent. Thus, the dot product becomes $|(k\mathbf{y})\cdot \mathbf{y}| = |k| |\mathbf{y}\cdot\mathbf{y}| = |k|(\mathbf{y}\cdot\mathbf{y}) = |k| |\mathbf{y}|^2$ . The right hand side becomes $|\mathbf{ky}| |\mathbf{y}| = |k||\mathbf{y}|^2$ . It is clear that the second case also results in a strict equality.

Finally, we consider the case in which the vectors $\mathbf{x}$ and $\mathbf{y}$ are linearly independent. Then, we have that $\mathbf{x} + k\mathbf{y}\ne \mathbf{0}$ , so that $0 < |\mathbf{x} - k\mathbf{y}|^2$ . But this is equal to $(\mathbf{x} - k\mathbf{y})\cdot (\mathbf{x} - k\mathbf{y})$ , and by the distributivity property of the dot product, this we have

$\begin{aligned} 0 &< \mathbf{x}\cdot\mathbf{x} - 2k(\mathbf{x}\cdot\mathbf{y}) + k^2(\mathbf{y}\cdot\mathbf{y}) \\ &= |\mathbf{x}|^2 - 2k(\mathbf{x}\cdot\mathbf{y}) + k^2 |\mathbf{y}|^2 \end{aligned}$

Now, taking the $-2k(\mathbf{x}\cdot\mathbf{y})$ to the other side, we have

$2k(\mathbf{x}\cdot\mathbf{y}) < |\mathbf{x}|^2 + k^2|\mathbf{y}|^2$

We know that this is true for any $k\in\mathbb{R}$ . Furthermore, when we are simply comparing vectors, we are not indirectly referring to any specific value of $k$ . Hence, we can choose a specific value of $k$ in order to simplify our inequality further. Setting $k = \frac{(\mathbf{x}\cdot\mathbf{y})}{|\mathbf{y}|^2}$ , we obtain

$2\frac{(\mathbf{x}\cdot\mathbf{y})}{|\mathbf{y}|^2}(\mathbf{x}\cdot\mathbf{y}) < |\mathbf{x}|^2 + (\frac{(\mathbf{x}\cdot\mathbf{y})}{|\mathbf{y}|^2})^2|\mathbf{y}|^2$

Which simplifies to

$2\frac{(\mathbf{x}\cdot\mathbf{y})^2}{|\mathbf{y}|^2} < |\mathbf{x}|^2 + \frac{(\mathbf{x}\cdot\mathbf{y})^2}{|\mathbf{y}|^2}$

$2(\mathbf{x}\cdot\mathbf{y})^2 < |\mathbf{x}|^2 |\mathbf{y}|^2+ (\mathbf{x}\cdot\mathbf{y})^2$

And finally taking subtracting $(\mathbf{x}\cdot\mathbf{y})^2$ from both sides, we obtain

$(\mathbf{x}\cdot\mathbf{y})^2 < |\mathbf{x}|^2 |\mathbf{y}|^2$

$|\mathbf{x}\cdot\mathbf{y}| < |\mathbf{x}| |\mathbf{y}|$

Our three cases have covered all possibilities, and therefore we have that

$|\mathbf{x}\cdot\mathbf{y}| \leq |\mathbf{x}| |\mathbf{y}|$

As desired.

Properties

Following from the above proof, if the two vectors are linearly independent, then we have a strict inequality. On the other hand, if the two vectors are linearly dependent, we have a strict equality.