Vectors and Linear Combinations

Vectors

Vectors are one of the main objects that we use in linear algebra. For now they are defined as elements of $\mathbb{R}^m, m\in \mathbb{N}$ which represents the $m$ dimensional space. So a vector $\textbf{v}\in \R^m$ is a sequence of $m$ real numbers.

This might bring up the question: How does $\R^0$ look like? In $\R^0$ we have only one element: the empty sequence $()$ of real numbers.

Vectors in a Coordinate System

Thinking of vectors as a way to move around in the $m$ dimensional space is a good way of understanding vectors. This might seem easy on a 2 dimensional example. However visualizing vectors and other objects we use in linear algebra like matrices and vector spaces is going to become more difficult as the number of dimensions increases.

v = Drag the tip of the arrow.

Vector Addition and Scalar Multiplication

You can imagine the vector addition as putting the vectors end to end, in order to reach where you want to go. On the other hand, scalar multiplication is just deciding how many of the same vector you want to use. Multiplying by $-1$ changes the direction.

Addition: $\textbf{v},\textbf{u} \in \mathbb{R}^m: \textbf{v} + \textbf{u} = \begin{bmatrix} v_1 \\ \vdots \\ v_m \end{bmatrix} + \begin{bmatrix} u_1 \\ \vdots \\ u_m \end{bmatrix} = \begin{bmatrix} v_1+u_1 \\ \vdots \\ v_m+u_m \end{bmatrix} \in \mathbb{R}^m$

v = u = v + u =

Scalar Multiplication: $\lambda \in \mathbb{R}, \textbf{v} \in \mathbb{R}^m: \lambda \cdot \textbf{v} = \lambda \cdot \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\v_m \end{bmatrix} = \begin{bmatrix} \lambda \cdot v_1 \\ \lambda \cdot v_2 \\ \vdots \\\lambda \cdot v_m \end{bmatrix} \in \mathbb{R}^m$

λ = v = λ · v =

Linear Combinations

Combine vector addition and scalar multiplication.

Let $\lambda_1, \dots, \lambda_m \in \mathbb{R} \text{ be m real numbers and}, \textbf{v}_1, \dots, \textbf{v}_m \in \mathbb{R}^n$ be $m$ vectors in $\R^n$ . Then:

\lambda_1 \cdot \textbf{v}_1 + \dots + \lambda_m \cdot \textbf{v}_m

is a linear combination of the vectors $\textbf{v}_1, \dots, \textbf{v}_m$ . Note that the result is again a vector in the same vector space (i.e. the resulting vector has the same dimension). So we can produce vectors using other vectors!

A natural question is whether or not we can produce all vectors by using a given set of vectors. Think in 2 dimensions. It might be relatively easy to see that the vectors:

\begin{pmatrix} 1 \\ 0 \end{pmatrix} , \begin{pmatrix} 0 \\ 1 \end{pmatrix}

can be used to produce all 2 dimensional vectors. If you want to write an arbitrary two dimensional vector

\begin{pmatrix} v_1 \\ v_2 \end{pmatrix}

as a linear combination of these two vectors -where values of $v_1$ and $v_2$ are known to you- you write:

\lambda \begin{pmatrix} 1 \\ 0 \end{pmatrix} + \mu \begin{pmatrix} 0 \\ 1 \end{pmatrix} = \begin{pmatrix} v_1 \\ v_2 \end{pmatrix} \text{ for } \lambda,\mu \in \R

and try to find the suited coefficients. In this case $\lambda = v_1$ and $\mu = v_2$ .

Example

Try to write the vector $\begin{pmatrix} 4 \\ 5 \end{pmatrix}$ using the vectors $\begin{pmatrix} 1 \\ 2 \end{pmatrix}, \begin{pmatrix} 3 \\ 0 \end{pmatrix}$ .

Solution: We are looking for real numbers $\lambda$ and $\mu$ such that

\lambda \begin{pmatrix} 1 \\ 2 \end{pmatrix} + \mu \begin{pmatrix} 3 \\ 0 \end{pmatrix} = \begin{pmatrix} 4 \\ 5 \end{pmatrix}

. If you do scalar multiplication and vector addition as above, you get

\begin{pmatrix} \lambda \cdot 1 + \mu \cdot 3 \\ \lambda \cdot 2 - \mu \cdot 0 \end{pmatrix} = \begin{pmatrix} 4 \\ 5 \end{pmatrix}

In order for these vectors to be equal, all elements need to be pairwise equal that is:

\begin{aligned} \lambda \cdot 1 + \mu \cdot 3 &= 4 \\ \lambda \cdot 2 - \mu \cdot 0 &= 5 \end{aligned}

Solving this linear system of equations (LSE) gives us $\lambda = 2.5$ and $\mu = 1.5$ .

Row & Column Pictures

We can distinguish between the column picture and the row picture when speaking of a linear combination. As an example you try to write an arbitrary vector

\textbf{u} = \begin{pmatrix} u_1 \\ u_2 \end{pmatrix} \in \R^2

as the linear combination of

\textbf{v} = \begin{pmatrix} 3 \\ 1 \end{pmatrix}, \textbf{t} = \begin{pmatrix} 1 \\ 2 \end{pmatrix}

Then you follow the same steps as above, try to find coefficients $\lambda, \mu \in \R$ such that

\lambda \begin{pmatrix} 3 \\ 1 \end{pmatrix} + \mu \begin{pmatrix} 1 \\ 2 \end{pmatrix} = \begin{pmatrix} u_1 \\ u_2 \end{pmatrix}

Now you can think of this linear combination in two ways:

The Column Picture

v = t = u = λ = , μ =

This approach tells you that you can build the vector $\textbf{u}$ by adding the multiples of the vectors $\textbf{v}$ and $\textbf{t}$ end to end. You can use the vectors as building blocks or steps to get to the vector you want to reach. In the process of doing this, how many times you used which vector gives you $\lambda$ and $\mu$ . See how the green and orange lines resemble new coordinate axes.

Now imagine that the two vectors were parallel to each other. When you put both to the origin they would be on a single line. This means no matter how you combine those vectors to get to some point in your coordinate system, you will stay on that line. Therefore in that scenario you cannot produce all vectors in your coordinate system using the ones you have.

Note that two vectors are parallel when they are a scalar multiple of each other. In this case we call them collinear.

Try to change the values of the vectors $\mathbf{v}$ and $\mathbf{t}$ above to make them collinear. See how the two lines that are spanned by the two vectors collapse to a single line. In that scenario those two vectors can only represent the vectors that are on that line.

The Row Picture

v t u λ + μ = λ + μ = λ = , μ =

The row picture actually serves the same purpose of finding $\lambda$ and $\mu$ . However this time the focus is on the elements of the individual vectors. See how the axes now represent $\lambda$ and $\mu$ . You write the linear combination in an extended form:

\begin{aligned} 3\lambda + 1\mu &= u_1 \\ 1 \lambda + 2\mu &= u_2 \end{aligned}

And focus on the lines that are represented by each equation. The intersection of these lines give you the values for $\lambda$ and $\mu$ that satisfy both equations and hence the whole linear system of equations (LSE). We are going to deal wit LSE’s extensively in the following weeks.

Try to make the vectors $\mathbf{v}$ and $\mathbf{t}$ collinear again. See how this makes the two lines parallel, indicating that there exists no intersection and hence no solution for $\lambda$ and $\mu$ that satisfy both equations in that case. One special case is when two lines of equations are a multiple of each other. Then two lines collapse to a single line, meaning that there are infinitely many solutions: all $\lambda$ and $\mu$ on that single line satisfy both equations.

Affine, Conic, and Convex Combinations

We can observe different kinds of special linear combinations. For a linear combination of n vectors where $\lambda_1\dots\lambda_m \in \R$ and $\textbf{v}_1, \dots, \textbf{v}_m \in \R^n$

\lambda_1 \textbf{v}_1 + \cdots + \lambda_m \textbf{v}_m

We say the linear combination is affine, conic, or convex if it satisfies the following corresponding conditions:

Affine: $\lambda_1 + \dots + \lambda_m = 1$
Conic: $\forall i: \lambda_i \ge 0$
Convex: Affine and Conic

Affine Combination of Two Vectors

We can take a closer look to affine linear combinations to better understand why they form a line for two vectors $\textbf{v}_1$ and $\textbf{v}_2$ in $\R^2$ .

\begin{aligned} &\lambda_1 \cdot \textbf{v}_1 + \lambda_2 \cdot \textbf{v}_2 && (\text{since }\lambda_1 + \lambda_2 = 1) \\ &= \lambda_1 \cdot \textbf{v}_1 + (1-\lambda_1)\cdot \textbf{v}_2 \\ &= \lambda_1 \cdot \textbf{v}_1 + 1\cdot \textbf{v}_2 -\lambda_1 \cdot \textbf{v}_2 \\ &= \textbf{v}_2 + \lambda_1(\textbf{v}_1 - \textbf{v}_2) \end{aligned}

The last line of equation tells us to go to $v_2$ first and then use any scalar multiple of the vector $v_1-v_2$ to go somewhere on the line that goes through $v_2$ and is parallel to $v_1-v_2$ .