Inner Product Space

$\spadesuit$ Piecewise Continuous Function $\spadesuit$

In this section, we introduce a vector space which is very different from spce vector. First let $C(a, b)$ be the set of continous functions on $(a,b)$ . Then let $PC(a,b)$ be the set of piecewise continuous function). Now for the function $f(t)$ to be piecewise continuous on $I$ , if the followings are satisfied.

is continuous on except finitely many points.
There exist a left-hand limit and a right-hand limit at the discontinuity $t_{0}$ of .

We now define an addition and a scalar multiplication on $C(a, b)$ and $PC(a,b)$ .
For $f,g$ in or , 1. $f+g$ is defined as a function which has the value $f(x)+g(x)$ at $x$ .
2. $\alpha f$ is defined as a function which has the value $\alpha f(x)$ at $x$ .

Example 1..3

For

, find $f+g, \frac{1}{2}f, 2g$ .

Answer $(f+g)(x) = f(x) + g(x) = x + x^2 ,$
$(\frac{1}{2}f)(x) = \frac{1}{2}f(x) = \frac{x}{2} ,$
$(2g)(x) = 2g(x) = 2x^2 .$ $\blacksquare$

With this addition and scalar multiplication,the properties 1 thru 9 to be a vector space is satisfied in $C(a, b)$ and $PC(a,b)$ .

Theorem 1..2

Show that

and

are vector space.

From now on we can call $f(x)$ and $g(x)$ vectors. It looks very different form a geomtric vectors. But it is a vector. You might notice these vectors might not have a magnitude or direction. For any vector space we can define a magnitude is called normed vector space. In this section, we consider a normed vector space which admits a dot product. $\spadesuit$ Inner Product $\spadesuit$

For geometric vectors, addition , scalar multiplicaiton ,and inner product are basic.

We have already studied an addition and a scalar multiplication of geometric vectors. So, here we study an inner product.

Consider nonzero vector A and B and the angle $\theta(0 \leq \theta \leq \pi)$ between A and B. Then the dot product of A and B is defined as $\Vert{\bf A}\Vert\Vert{\bf B}\Vert\cos{\theta}$ . In other words,

$\displaystyle {\bf A} \cdot {\bf B} = \Vert{\bf A}\Vert\Vert{\bf B}\Vert\cos{\theta}$

If either A or B is 0, then we define ${\bf A} \cdot {\bf B} = 0$ .

Up to now, we have tried to generalize the addition and the scalar multiplication. Now we try to generalize an inner product.

Definition 1..1

For two vectors ${\bf v}_{1} and {\bf v}_{2}$ , A real number ${\bf v}_{1}\cdot{\bf v}_{2}$ is defined and satisfies the following properties, ${\bf v}_{1}\cdot{\bf v}_{2}$ is called an inner product of vectors ${\bf v}_{1}$ and ${\bf v}_{2}$

For any vectors ${\bf v}, {\bf v}_{1},{\bf v}_{2},{\bf v}_{3}$ and real numbers $\alpha, \beta$ , the followings are hold.

$(\alpha \vec{v}_{1} + \beta \vec{v}_{2}) \cdot (\vec{v}_{3}) = \alpha (\vec{v}_{1} \cdot \vec{v}_{3}) + \beta (\vec{v}_{2} \cdot \vec{v}_{3})$ (linearlity)
$\vec{v}_{1} \cdot \vec{v}_{2} = \vec{v}_{2} \cdot \vec{v}_{1}$ symmetric
$\vec{v} \cdot \vec{v} \geq 0, \vec{v} \cdot \vec{v} = 0$ and $\vec{v} = \vec{0}$ are equivalent . (positive definite)

Example 1..4

For geometric vectors A and B, The inner product A $\cdot$ B can be thought of

$\displaystyle {\bf A} \cdot {\bf B} = \vert{\bf A}\vert \vert{\bf B}\vert \cos{\theta} .$

Here, $\Vert{\bf A}\Vert$ and $\vert{\bf B}\vert$ are the magnitude of A and B. $\theta$ represents a smaller angle between A and B.

**Figure 1.2:** Scalar multiple
$\includegraphics[width=6cm]{LALG/Fig1-2.eps}$

Example 1..5

${\bf A} \cdot {\bf B} = \vert{\bf A}\vert \vert{\bf B}\vert \cos{\theta}$ satisfies the definition of inner product.

Answer Let $\theta$ be the angle between A and B. Let $\gamma$ be the angle between A and A+B. Let $\omega$ be the angle between A and C
Then
1.

$\displaystyle \vert{\bf A+B}\vert\cos{\gamma} = \vert{\bf A}\vert + \vert{\bf B}\vert\cos{\theta},$

$\displaystyle \vert{\bf A+B}\vert\sin{\gamma} = \vert{\bf B}\vert\sin{\theta}.$

Thus,

$\displaystyle ({\bf A +B})\cdot {\bf C}$	$\displaystyle =$	$\displaystyle \vert{\bf A}+{\bf B}\vert \vert{\bf C}\vert \cos({\omega} - {\gamma})$
	$\displaystyle =$	$\displaystyle \vert{\bf A}+{\bf B}\vert \vert{\bf C}\vert(\cos{\omega}\cos{\gamma} + \sin{\omega}\sin{\gamma})$
	$\displaystyle =$	$\displaystyle \vert{\bf A}\vert \vert{\bf C}\vert \cos{\omega} + \vert{\bf B}\vert \vert{\bf C}\vert \cos({\theta} - {\omega})$
	$\displaystyle =$	$\displaystyle {\bf A} \cdot {\bf C} + {\bf B} \cdot {\bf C}.$

Also, for $\alpha \neq 0$ , we have

$\displaystyle \alpha({\bf A}\cdot{\bf B}) = \alpha\vert{\bf A}\vert \vert{\bf B}\vert \cos{\theta} = (\alpha{\bf A})\cdot {\bf B}.$

We are able to show linearlity.
2.

$\displaystyle {\bf A}\cdot{\bf B} = \vert{\bf A}\vert \vert{\bf B}\vert \cos{\theta} = \vert{\bf B}\vert \vert{\bf A}\vert \cos{\theta} = {\bf B}\cdot{\bf A}.$

3. ${\bf A}\cdot{\bf A} = \vert{\bf A}\vert^2$ , Thus, ${\bf A}\cdot{\bf A} = 0$ and A = 0 are equivalent. $\blacksquare$

Example 1..6

Define an inner product on ${ R}^3$ .

Answer Let $(a_{1},a_{2},a_{3}), (b_{1},b_{2},b_{3})$ be elements of ${ R}^3$ . Then

$\displaystyle (a_{1},a_{2},a_{3}) \cdot (b_{1},b_{2},b_{3}) = a_{1}b_{1} + a_{2}b_{2} + a_{3}b_{3}$

is an inner product because if ${\bf v}_{1} = (a_{1},a_{2},a_{3}), {\bf v}_{2} = (b_{1},b_{2},b_{3}), {\bf v}_{3} = (c_{1},c_{2},c_{3}) \in { R}^{3}, \alpha, \beta \in R$ , then we have 1.

$\displaystyle (\alpha {\bf v}_{1} + \beta {\bf v}_{2})\cdot {\bf v}_{3}$	$\displaystyle =$	$\displaystyle (\alpha a_{1} + \beta b_{1}, \alpha a_{2} + \beta b_{2}, \alpha a_{3} + \beta b_{3}) \cdot (c_{1}, c_{2}, c_{3})$
	$\displaystyle =$	$\displaystyle (\alpha a_{1} + \beta b_{1})c_{1} + (\alpha a_{2} + \beta b_{2})c_{2} + (\alpha a_{3} + \beta b_{3})c_{3}$
	$\displaystyle =$	$\displaystyle \alpha(a_{1}c_{1} + a_{2}c_{2} + a_{3}c_{3}) + \beta(b_{1}c_{1} + b_{2}c_{2} + b_{3}c_{3})$
	$\displaystyle =$	$\displaystyle \alpha(a_{1},a_{2},a_{3}) \cdot (c_{1}.c_{2},c_{3}) + \beta(b_{1},b_{2},b_{3}) \cdot (c_{1},c_{2},c_{3})$
	$\displaystyle =$	$\displaystyle \alpha {\bf v}_{1} \cdot {\bf v}_{3} + \beta {\bf v}_{2} \cdot {\bf v}_{3}.$

$\displaystyle {\bf v}_{1} \cdot {\bf v}_{2}$	$\displaystyle =$	$\displaystyle (a_{1},a_{2},a_{3}) \cdot (b_{1},b_{2},b_{3})$
	$\displaystyle =$	$\displaystyle a_{1}b_{1} + a_{2}b_{2} + a_{3}b_{3}$
	$\displaystyle =$	$\displaystyle b_{1}a_{1} + b_{2}a_{2} + b_{3}a_{3}$
	$\displaystyle =$	$\displaystyle (b_{1},b_{2},b_{3}) \cdot (a_{1},a_{2},a_{3})$
	$\displaystyle =$	$\displaystyle {\bf v}_{2} \cdot {\bf v}_{1}.$

$\displaystyle {\bf v}_{1} \cdot {\bf v}_{1} = a_{1}^2 + a_{2}^2 + a_{3}^2 \geq 0$

$\displaystyle {\bf v}_{1} \cdot {\bf v}_{1} = 0$	$\displaystyle \Leftrightarrow$	$\displaystyle a_{1}^2 + a_{2}^2 + a_{3}^2 = 0$
	$\displaystyle \Leftrightarrow$	$\displaystyle a_{1} = a_{2} = a_{3} = 0. \ensuremath{ \blacksquare}$

Definition 1..2

Suppose that $f(x),g(x)$

are elements of $PC[a,b]$

. Then the inner product $(f,g)$

is given by the followings:.

$\displaystyle (f,g) = \int_{a}^{b}f(x)g(x)dx .$

(notice) This satisfies the properties of the inner product $1, 2, 3$

Example 1..7

For

are in

, find

Answer

$\displaystyle (f,g) = \int_{0}^{2}(x)(x^2)dx = \frac{x^4}{4}\mid_{0}^{2} = 4. \ensuremath{ \blacksquare}$

If an inner product is defined on a vector space, then we can define a norm.

Definition 1..3

The norm of a vector v is written as $\Vert{\bf v}\Vert _{2}$ and represented by the followings:

$\displaystyle \Vert{\bf v}\Vert _{2} = \sqrt{{\bf v}\cdot{\bf v}}.$

Then by the properties of inner product, $l_{2}$ norm has the following properties.

For any vectors ${\mathbf x}, {\mathbf y} \in {\cal R}^{n}$ and any real number $\alpha$ , we have

$\Vert{\mathbf x}\Vert _{2} \geq 0, \Vert{\mathbf x}\Vert _{2} = 0$ and ${\mathbf x} = 0$ are equivalent.
$\Vert \alpha {\mathbf x}\Vert _{2} = \vert\alpha\vert\Vert{\mathbf x}\Vert _{2}$
$\Vert{\mathbf x} + {\mathbf y}\Vert _{2} \leq \Vert{\mathbf x}\Vert _{2} + \Vert{\mathbf y}\Vert _{2}$

For example in the geometric vector space, $\Vert{\bf A}\Vert _{2} = \sqrt{{\bf A}\cdot{\bf A}} = \vert{\bf A}\vert$ . Thus it is the same as the length of A. For the space vectors,

$\displaystyle \Vert(a_{1},a_{2},a_{3})\Vert _{2} = \sqrt{a_{1}^2+a_{2}^2+a_{3}^2}$

which can be considered as the shortest distance to $(a_{1},a_{2},a_{3})$ . For the function space $PC[a,b]$

, we have

$\displaystyle \Vert f\Vert _{2} = \{\int_{a}^{b}[f(x)]^2dx\}^{1/2}.$

Other than $l_{2}$ norm, it is often used in ${\cal R}^{n}$ , we have $l_{\infty}$ norm.

Definition 1..4

For a vector ${\mathbf x} = (x_{1},x_{2},\ldots,x_{n})$ , $l_{\infty}$ norm(norm) is written as $\Vert{\mathbf x}\Vert _{\infty}$ and defined as follows:.

$\displaystyle \Vert{\mathbf x}\Vert _{\infty} = \max_{1 \leq i \leq n}\vert x_{i}\vert.$

Example 1..8

Find a $l_{2}$ norm and $l_{\infty}$ norm of ${\mathbf x} = (-1,1,-2)$ .

Answer

$\displaystyle \Vert{\mathbf x}\Vert _{2}$	$\displaystyle =$	$\displaystyle \sqrt{(-1,1,-2) \cdot (-1,1,-2)} = \sqrt{1 + 1 + 4} = \sqrt{6}$
$\displaystyle \Vert{\mathbf x}\Vert _{\infty}$	$\displaystyle =$	$\displaystyle \max_{1 \leq i \leq 3}\vert x_{i}\vert = \max\{1, 1, 2\} = 2$

$l_{\infty}$ norm and $l_{2}$ norm have the following properties.

For any vectors ${\mathbf x}, {\mathbf y} \in {\cal R}^{n}$ and any real number $\alpha$ , we have

$\Vert{\mathbf x}\Vert \geq 0, \Vert{\mathbf x}\Vert = 0$ and ${\mathbf x} = 0$ equivalent.
$\Vert\alpha {\mathbf x}\Vert = \vert\alpha\vert\Vert{\mathbf x}\Vert$
$\Vert{\mathbf x} + {\mathbf y}\Vert \leq \Vert{\mathbf x}\Vert + \Vert{\mathbf y}\Vert$

$\spadesuit$ Orthogonal $\spadesuit$

If any two geometric vectors are orthogonal, then、 ${\bf A} \cdot {\bf B} = \vert{\bf A}\vert \vert{\bf B}\vert \cos{\theta}$ で $\cos{\theta} = 0$ . Thus, the inner product is 0、On the other hand, if ${\bf A},{\bf B}$ are not zero vectors and the inner product is 0, then $\cos{\theta} = 0$ and ${\bf A}$ and ${\bf B}$ are orthogonal.

In this way, when the inner product is defined in a vector space, not only norm but the concept of orthogonal can be introduced.

Definition 1..5

Two vectors ${\bf v}_{1}$ and ${\bf v}_{2}$ are said to be orthogonal if ${\bf v}_{1} \cdot {\bf v}_{2} = 0$ . A set of vectors whose elements are orthogonal to each other is called orthogonal system.

In the geometric vector space by 1.2 , an inner product of directed lines ${\bf A}\cdot{\bf B}$ is given by ${\bf A} \cdot {\bf B} = \vert{\bf A}\vert \vert{\bf B}\vert \cos{\theta}$ . For if nonzero directed lines A and B, ${\bf A} \cdot {\bf B} = 0$ , then $\theta = 90^{\circ}$ . Thus, A and B are orthogonal.

In the 3D vector space, the inner product of $(a_{1},a_{2},a_{3})$ and $(b_{1},b_{2},b_{3})$ is given by $a_{1}b_{1}+a_{2}b_{2}+a_{3}b_{3}$ according to 1.2. If we think of 3D vector as a geometric vector, then

$\displaystyle (a_{1}, a_{2}, a_{3}) \cdot (b_{1}, b_{2}, b_{3}) = \Vert(a_{1}, a_{2}, a_{3})\Vert \Vert(b_{1}, b_{2}, b_{3})\Vert \cos{\theta}.$

Then, we have

$\displaystyle \cos{\theta} = \frac{a_{1}b_{1} + a_{2}b_{2} + a_{3}b_{3}}{\Vert(a_{1},a_{2},a_{3})\Vert \Vert(b_{1},b_{2},b_{3})\Vert}$

Thus, we can find the angle between two 3D vectors

Example 1..9

Find the angle between ${\bf A} = (1,1,2)$ and ${\bf B} = (-1,2,1)$ .

Answer

$\displaystyle \cos{\theta} = \frac{a_{1}b_{1} + a_{2}b_{2} + a_{3}b_{3}}{\Vert(... ...Vert(b_{1},b_{2},b_{3})\Vert} = \frac{-1+2+2}{\sqrt{6}\sqrt{6}} = \frac{1}{2}.$

To find the $\theta$ in the interval $80,\frac{\pi}{2}]$ satisfying the above, we have $\displaystyle{\theta = \frac{\pi}{3}}$ . $\blacksquare$

$\spadesuit$ Equation of Plane $\spadesuit$

**Figure 1.3:** Equation of Plane
$\includegraphics[width=8cm]{LALG/Fig1-3.eps}$

Consider a coordinate axis in the space and imagine a plane. A normal vector is a vector orthogonal to any tangent vector. If we use a inner product, we can find an equation of this plane. Put a point $(a_{1},a_{2},a_{3})$ on the plane. Let N = $(A,B,C)$ be the normal vector to the plane. Let ${\bf r}_{0}$ be the position vector connecting the origin $(0,0,0)$ and $(a_{1}, a_{2},$ $a_{3})$ . Now let r be a vector connecting the origin and the point different from $(a_{1},a_{2},a_{3})$ . Then the vector ${\bf r} - {\bf r}_{0}$ is on the plane and the angle between ${\bf r} - {\bf r}_{0}$ and N is $90^{\circ}$ . Thus, we have $({\bf r} - {\bf r}_{0}) \cdot {\bf N} = 0$ . This is the equation of the plane.

Example 1..10

Find the equation of the plane going thru the point $(3,-1,4)$

and the normal vector is $(-1,1,2)$

Answer Let the position vector r be ( $x, y, z$ ). Then the equation of the plane is

$\displaystyle (x-3, y+1, z-4) \cdot (-1,1,2) = 0.$

Thus, we have

$\displaystyle -(x-3) + (y+1) + 2(z-4) = 0$

$\displaystyle -x + y + 2z = 4. \ensuremath{ \blacksquare}$

Example 1..11

Show that a function $\sin{x}$ and a function $\cos{x}$ is orthogonal on $[-\pi,\pi]$ . But nor on $\displaystyle{[0,\frac{\pi}{4}]}$ .

Answer $[-\pi,\pi]$ で

$\displaystyle (\sin{x},\cos{x}) = \int_{-\pi}^{\pi}\sin{x}\cos{x}dx = \frac{\sin^{2}{x}}{2}\mid_{-\pi}^{\pi} = \frac{\sin^{2}{\pi} - \sin^{2}{(-\pi})}{2} = 0 .$

$\displaystyle{[0,\frac{\pi}{4}]}$ で

$\displaystyle (\sin{x},\cos{x}) = \int_{0}^{\frac{\pi}{4}}\sin{x}\cos{x}dx = \f... ...\frac{1}{2}(\frac{\sqrt{2}}{2})^{2} = \frac{1}{4}. \ensuremath{ \blacksquare}$

In function space, an orthogonal doen not mean perpendicular.

A unit vector ${\bf u}$ is a vector whose magnitude is 1. Given nonzero vector A, we can find a unit vector with the same direction as A. To do so, simply divide A by its magnitude $\Vert{\bf A}\Vert$ . For a general vector, we might try the same thing, that is, divide the vector v by its norm $\Vert{\bf v}\Vert$ . To find a unit vector by dividing its norm is called normalization. A set of vectors whose elements are all normalized is called orthonormal system. As an example of orthonormal system, we have seen {i,j,k}.

Example 1..12

Show that the function system ${\{\cos{mx}\}}_{m=0}^{\infty}$ is an otthogonal system on $[0,\pi]$ . Then find a correspondin orthonormal system.

Answer For $m \neq n$ , we have

$\displaystyle (\cos{mx},\cos{nx})$	$\displaystyle =$	$\displaystyle \int_{0}^{\pi}\cos{mx}\cos{nx}dx$
	$\displaystyle =$	$\displaystyle \frac{1}{2}\int_{0}^{\pi}[\cos{(m+n)x} + \cos{(m-n)x}]dx$
	$\displaystyle =$	$\displaystyle \frac{1}{2}[\frac{\sin{(m+n)x}}{(m+n)} + \frac{\sin{(m-n)x}}{(m-n)}] \mid_{0}^{\pi} = 0.$

For

, $\cos{mx} = \cos{0} = 1$ implies that

$\displaystyle \Vert 1 \Vert = [ \int_{0}^{\pi}(1)^2 dx]^{1/2} = \sqrt{\pi}.$

For $m \neq 0$ , we have

$\displaystyle \Vert\cos{mx}\Vert = [ \int_{0}^{\pi}\cos^2{mx} dx]^{1/2} = \sqrt{\frac{\pi}{2}}.$

Thus the required orthonomal system is as follows:

$\displaystyle \{\frac{1}{\sqrt{\pi}}, \frac{\sqrt{2}}{\sqrt{\pi}}\cos{x}, \cdots , \frac{\sqrt{2}}{\sqrt{\pi}}\cos{mx},\cdots \}. \ensuremath{ \blacksquare}$

$\spadesuit$ Fourier Series $\spadesuit$

Using the orthnormal system mentioned above, we can represent the piecewise continuous function $f(x) \in [0,L]$ as follows:

$\displaystyle f(x) \sim \frac{a_{0}}{2} + \sum_{n=1}^{\infty}a_{n}\cos{\frac{n\pi x}{L}}$

Here,

$\displaystyle a_{n} = \frac{2}{L}\int_{0}^{\infty}f(x)\cos{\frac{n\pi x}{L}}dx, n = 0,1,2,\ldots,$

The right hand side of the first expression is called Fourier series of $f(x)$

. $a_{n}$ is called Fourier coefficient.

Example 1..13

Show the norm on ${\bf v}_{1}$ and ${\bf v}_{2}$ satisfies the triangle inequality

$\displaystyle \Vert{\bf v}_{1} + {\bf v}_{2}\Vert \leq \Vert{\bf v}_{1} \Vert + \Vert {\bf v}_{2}\Vert$

Answer

$\displaystyle \Vert{\bf v}_{1} + {\bf v}_{2}\Vert^{2}$	$\displaystyle =$	$\displaystyle ({\bf v}_{1} + {\bf v}_{2})\cdot ({\bf v}_{1} + {\bf v}_{2})$
		$\displaystyle \Vert{\bf v}_{1}\Vert^{2} + 2 {\bf v}_{1} \cdot {\bf v}_{2} + \Vert{\bf v}_{2}\Vert^{2}$

Here we use Cauchy-Schwarz inequality (Exercise1.2). Then

$\displaystyle 2 {\bf v}_{1} \cdot {\bf v}_{2} \leq 2\Vert{\bf v}_{1}\Vert \Vert{\bf v}_{2}\Vert$

Thus,

$\displaystyle \Vert{\bf v}_{1} + {\bf v}_{2}\Vert \leq \Vert{\bf v}_{1} \Vert + \Vert {\bf v}_{2}\Vert \ensuremath{ \blacksquare}$

From this, distance satifies the followings:.

$\displaystyle \Vert{\bf v}_{1} - {\bf v}_{2}\Vert \leq \Vert{\bf v}_{1} - {\bf v}_{3}\Vert + \Vert{\bf v}_{3} - {\bf v}_{2}\Vert .$

Thus, $l_{2}$ norm just behaves like the distance we use.

Example 1..14

Find the distance between $x$

and

Answer

$\displaystyle \Vert x - x^2\Vert = [\int_{0}^{2}(x -x^2)^2 dx]^{1/2} = \frac{4}{\sqrt{15}} . \ensuremath{ \blacksquare}$

Exercise1-4

1. For vectors ${\bf A} = (-1,3,1)$ and ${\bf B} = (2,4,-3)$ , find the followings: (a) $\vert{\bf B}\vert$ (b) ${\bf A}\cdot{\bf B}$ (c) Angle between A and B (d) Unit vector in the dierction of A