Docsity
Docsity

Prepare for your exams
Prepare for your exams

Study with the several resources on Docsity


Earn points to download
Earn points to download

Earn points by helping other students or get them with a premium plan


Guidelines and tips
Guidelines and tips

Linear Independence, Basis, and Dimension in Linear Algebra, Lecture notes of Machine Learning

lecture slides for introduction to machine leraning

Typology: Lecture notes

2018/2019

Uploaded on 11/01/2019

chetan-reddy
chetan-reddy 🇺🇸

9 documents

1 / 9

Toggle sidebar

This page cannot be seen from the preview

Don't miss anything!

bg1
pf3
pf4
pf5
pf8
pf9

Partial preview of the text

Download Linear Independence, Basis, and Dimension in Linear Algebra and more Lecture notes Machine Learning in PDF only on Docsity!

Linear Independence, Basis, and Dimension

Vectors x 1 , x 2 , · · · , x (^) n are linearly independent if no combination gives c 1 x 1 + c 2 x 2 + · · · + c (^) n xn = 0 except for c 1 = c 2 = · · · = c (^) n = 0. For real vectors, linear independence is equivalent to non-collinearity (algebraic vs. geometrical meaning) 0 vector cannot be independent with any non-zero vector v , because we always have c 0 + 0v = 0 for c 6 = 0. Column vectors v 1 , v 2 , · · · , v (^) n of matrix A (^) m⇥n are Linearly independent if N(A) only includes 0 ) r = n  m, no free variables Linearly dependent if 9 vector x 6 = 0 s.t. Ax = 0 ) r < n, there are n r free variables Vectors v 1 , v 2 , · · · , v (^) n span a space if that space consists of ALL linear combinations of those vectors. e.g., two vectors cannot span R 3 e.g., four non-coplanar vectors span R 3 , but they must be linearly dependent!

Linear Independence, Basis, and Dimension

Check if a set of vectors forms a basis

List them as the column vectors of A and perform Gaussian elimination, then count the number of pivots in the Echelon form.

In general, n vectors form a valid basis of Rn^ if the n ⇥ n matrix they are involved in is invertible (non-singular). There can be infinite number of di↵erent bases Common: number of vectors (rank) ) Dimension of the space

Example

A =

(^5) r = dim C (A) = 2

The first 2 columns are pivot columns, so {v 1 , v 2 } is a basis Since v 3 = v 1 + v 2 , {v 1 , v 3 } is also a basis Since v 4 = 2v 1 , {v 1 , v 4 } is NOT a basis

Four fundamental subspaces

We have already defined Column space C (A) ⇢ R m^ : dim C (A) = r Null space N(A) ⇢ R n^ : Kernel of A, dim N(A) = n r ”Nullity” Similarly, by switching the roles of columns and rows, Row space C (A T^ ) ⇢ R n Left null space N(A T^ ) = {x|A T^ x = 0 or x T^ A = 0} ⇢ R m

Properties of fundamental subspaces

Subspace C (A) C (AT^ ) N(A) N(AT^ ) Dimension r r n r m r Basis Pivot col. Pivot col. Setting each Do the same of A of AT^ free variables 1 for AT^ as and solve Ux = 0 solving N(A)