Download Advanced Linear Algebra: Transformations, Geometry, and Inverses and more Lecture notes Machine Learning in PDF only on Docsity!
Linear Transformation
๐ ๐๐ข = ๐๐ ๐ข , โ c
Restricted to LINEAR:
2
2
3 is NOT linear
๐ 0 = 0 cannot move the origin
Linear transformation is not restricted to vector space:
Matrix ๐ can act on functions, polynomials, matrices, etc.
For matrix ๐ such that ๐๐ฃ = ๐ข with ๐ฃ โ ๐น
7 and ๐ข โ ๐น
8 ,
T 8 ร 7 : R
n โ R
m is a linear transformation if
Geometrical Operations in 2D: R
2 ร R
2
๐Q =
๐
S =
cos ๐ โ sin ๐
sin ๐ cos ๐
๐
S๐
U =
cos(๐ + ๐) โ sin(๐ + ๐)
sin(๐ + ๐) cos(๐ + ๐)
= ๐
SWU
(Abelian) Lie Group
๐
S๐
XS = ๐ผ [๐
[,^ ๐
]^ =^0
2D rotation
๐
^ =
Inverse
รผ ๐
3 = ๐
รผ โ ๐
Xg
If โ ๐
Xg , then ๐
Xg ๐
3 = ๐
Xg ๐
Xg trivial!
โ Projection in 2D:
cos
3 ๐ cos ๐ sin ๐
cos ๐ sin ๐ sin
3 ๐
๐ถ ๐ or ๐ถ(๐
k )
Null space ๐(๐)? Column space ๐ถ(๐)?
( Hint: ๐ = ๐
k )
cos
3 ๐ cos ๐ sin ๐
rank = 1
AAACEXicbVDLTsJAFL3FF+ILdeHCzURC4oq0aKJLohuXmMgjoYRMhymdMJ02M7caQvgKP8GtfoA749YvcO2PWAoLAU9yk5Nz7r0z93ixFAZt+9vKra1vbG7ltws7u3v7B8XDo6aJEs14g0Uy0m2PGi6F4g0UKHk71pyGnuQtb3g79VuPXBsRqQccxbwb0oESvmAUU6lXPDGuFoMAqdbRE3GNUC4GHGmvWLIrdgaySpw5KdXykKHeK/64/YglIVfIJDWm49gxdsdUo2CSTwpuYnhM2ZAOeCeliobcdMfZARNSTpU+8SOdlkKSqX8nxjQ0ZhR6aWdIMTDL3lT8z+sk6F93x0LFCXLFZg/5iSQYkWkapC80ZyhHCwu9cFIo9wVlWqSfJyygmjJMQyykqTjLGaySZrXiXFSq95el2s0sHsjDKZzBOThwBTW4gzo0gMEEXuAV3qxn6936sD5nrTlrPnMMC7C+fgEUBZ3o^ s^!^ sin^ โ
AAACEXicbVDLTsJAFL3FF+ILdeHCzURC4oq0aKJLohuXmMgjoYRMhymdMO00M7caQvgKP8GtfoA749YvcO2PWAoLAU9yk5Nz7r0z93ixFAZt+9vKra1vbG7ltws7u3v7B8XDo6ZRiWa8wZRUuu1Rw6WIeAMFSt6ONaehJ3nLG95O/dYj10ao6AFHMe+GdBAJXzCKqdQrnjBXi0GAVGv1RFymjIsBR9orluyKnYGsEmdOSrU8ZKj3ij9uX7Ek5BEySY3pOHaM3THVKJjkk4KbGB5TNqQD3klpRENuuuPsgAkpp0qf+EqnFSHJ1L8TYxoaMwq9tDOkGJhlbyr+53US9K+7YxHFCfKIzR7yE0lQkWkapC80ZyhHCwu9cFIo9wVlWqSfJyygmjJMQyykqTjLGaySZrXiXFSq95el2s0sHsjDKZzBOThwBTW4gzo0gMEEXuAV3qxn6936sD5nrTlrPnMMC7C+fgHxJZ3T^ c^!^ cos^ โ
๐ ๐ or ๐(๐
k )
โ tan ๐
โ (Mirror) reflection in 2D: Householder transformation
cos 2 ๐ sin 2๐
sin 2 ๐ โ cos 2 ๐
Parallelogram Mirror
Column 1
Column 2
AAAB/nicbVDLSgNBEOz1GddX1KOXwRDwFHZF0GPQi8cI5gHJEmZnZ5Mxsw9meoWwBPwEr/oB3sSrv+LZH3GyycEkFjQUVd0z3eWnUmh0nG9rbX1jc2u7tGPv7u0fHJaPjls6yRTjTZbIRHV8qrkUMW+iQMk7qeI08iVv+6Pbqd9+4kqLJH7Accq9iA5iEQpG0UitHg450n654tScAmSVuHNSqZegQKNf/ukFCcsiHiOTVOuu66To5VShYJJP7F6meUrZiA5419CYRlx7ebHthFSNEpAwUaZiJIX6dyKnkdbjyDedEcWhXvam4n9eN8Pw2stFnGbIYzb7KMwkwYRMTyeBUJyhHC886EcTuxoIypQwyxM2pIoyNInZJhV3OYNV0rqouYbfX1bqN7N4oASncAbn4MIV1OEOGtAEBo/wAq/wZj1b79aH9TlrXbPmMyewAOvrF+LTlis=AAAB/nicbVDLSgNBEOz1GddX1KOXwRDwFHZF0GPQi8cI5gHJEmZnZ5Mxsw9meoWwBPwEr/oB3sSrv+LZH3GyycEkFjQUVd0z3eWnUmh0nG9rbX1jc2u7tGPv7u0fHJaPjls6yRTjTZbIRHV8qrkUMW+iQMk7qeI08iVv+6Pbqd9+4kqLJH7Accq9iA5iEQpG0UitHg450n654tScAmSVuHNSqZegQKNf/ukFCcsiHiOTVOuu66To5VShYJJP7F6meUrZiA5419CYRlx7ebHthFSNEpAwUaZiJIX6dyKnkdbjyDedEcWhXvam4n9eN8Pw2stFnGbIYzb7KMwkwYRMTyeBUJyhHC886EcTuxoIypQwyxM2pIoyNInZJhV3OYNV0rqouYbfX1bqN7N4oASncAbn4MIV1OEOGtAEBo/wAq/wZj1b79aH9TlrXbPmMyewAOvrF+LTlis=AAAB/nicbVDLSgNBEOz1GddX1KOXwRDwFHZF0GPQi8cI5gHJEmZnZ5Mxsw9meoWwBPwEr/oB3sSrv+LZH3GyycEkFjQUVd0z3eWnUmh0nG9rbX1jc2u7tGPv7u0fHJaPjls6yRTjTZbIRHV8qrkUMW+iQMk7qeI08iVv+6Pbqd9+4kqLJH7Accq9iA5iEQpG0UitHg450n654tScAmSVuHNSqZegQKNf/ukFCcsiHiOTVOuu66To5VShYJJP7F6meUrZiA5419CYRlx7ebHthFSNEpAwUaZiJIX6dyKnkdbjyDedEcWhXvam4n9eN8Pw2stFnGbIYzb7KMwkwYRMTyeBUJyhHC886EcTuxoIypQwyxM2pIoyNInZJhV3OYNV0rqouYbfX1bqN7N4oASncAbn4MIV1OEOGtAEBo/wAq/wZj1b79aH9TlrXbPmMyewAOvrF+LTlis=AAAB/nicbVDLSgMxFM34rOOr6tJNsBRclRkRdFl047KCfUA7lEwm08YmmSG5I5Sh4Ce41Q9wJ279Fdf+iGk7C9t64MLhnHuTe0+YCm7A876dtfWNza3t0o67u7d/cFg+Om6ZJNOUNWkiEt0JiWGCK9YEDoJ1Us2IDAVrh6Pbqd9+YtrwRD3AOGWBJAPFY04JWKnVgyED0i9XvJo3A14lfkEqqECjX/7pRQnNJFNABTGm63spBDnRwKlgE7eXGZYSOiID1rVUEclMkM+2neCqVSIcJ9qWAjxT/07kRBozlqHtlASGZtmbiv953Qzi6yDnKs2AKTr/KM4EhgRPT8cR14yCGC88GMqJW404oZrb5TEdEk0o2MRcm4q/nMEqaV3UfMvvLyv1myKfEjpFZ+gc+egK1dEdaqAmougRvaBX9OY8O+/Oh/M5b11zipkTtADn6xd8M5Xj^ โ
AAAB/nicbVDLSgMxFL3js46vqks3wVJwY5mpgm6Eopu6q2Af0A4lk8m0sUlmSDJCKQU/wa1+gDtx66+49kecTruwrQcuHM65N7n3+DFn2jjOt7Wyura+sZnbsrd3dvf28weHDR0litA6iXikWj7WlDNJ64YZTluxolj4nDb9we3Ebz5RpVkkH8wwpp7APclCRrBJpUb1ulw7u+vmC07JyYCWiTsjhUoOMtS6+Z9OEJFEUGkIx1q3XSc23ggrwwinY7uTaBpjMsA92k6pxIJqb5RtO0bFVAlQGKm0pEGZ+ndihIXWQ+GnnQKbvl70JuJ/Xjsx4ZU3YjJODJVk+lGYcGQiNDkdBUxRYvhw7kFfjO1iwDBRLF0ekT5WmJg0MTtNxV3MYJk0yiX3vFS+vyhUbqbxQA6O4QROwYVLqEAValAHAo/wAq/wZj1b79aH9TltXbFmM0cwB+vrF14klTo=^ H^ = 2P^ ^ I
Reverse problem: Constructing the matrix of linear transformation
go
2o
8o
โฏ 8 ร 7
T 8 ร 7 : R
n โ R
m
๐ = 1 , 2 , โฏn
Example 1:
Consider a linear operator L acting on R
2 as
Find the matrix of L with respect to the basis:
So we obtain, L =
T T
g
3
โ๐ฏ 3
๐ฏg
๐ฟ ๐ฏg = 2 ๐ฏg + (โ 1 )๐ฏ 3 ๐ฟ ๐ฏ 3 = 1 ๐ฏg + 0 ๐ฏ 3
Representation: state ร vector; operation ร matrix (quantum mechanics)
- Construct a (generalized) vector space
- Decompose a vector in terms of a basis: ๐ฅ = ๐ 2 ๐ฅ 2
- Act T on each base vector: ๐๐ฅ = ๐(๐ 2 ๐ฅ 2 ) = ๐ 2 ๐(๐ฅ 2 )
Differentiation operator ๐ด = ๐/๐๐ก
Basis ๐g = 1 , ๐ 3 = ๐ก, ๐f = ๐ก
3 , โฏ โฏ ๐ 7 = ๐ก
7 Xg , ๐ 7 Wg = ๐ก
7
Then, ๐ด๐ g
3
g
f
3
7
7 X 3 = ๐ โ 1 ๐ 7 Xg
7 Wg
7 Xg = ๐๐ 7
Reverse problem: Constructing the matrix of linear transformation
Differentiation operator ๐ด = ๐/๐๐ก on polynomial basis
๐ด 7 ร( 7 Wg): Pn+1โ Pn
Null space: ๐ด๐{ = ๐๐{/๐๐ก = 0 ,
Example : For ,
Taking derivative on an arbitrary constant = 0
k
One-sided Inverses:
- ๐ = ๐ = ๐ Full rank square matrix, ๐๐
Xg = ๐
Xg ๐ = ๐ผ (two-sided)
- ๐ = ๐ < ๐ Full column rank ( e.g. ๐ต = โซ ๐๐ก )
๐ ๐ต = {zero}, no independent columns, no free variable
k ๐ต)๐ฅ = 0 โน ๐(๐ต) = ๐(๐ต
k ๐ต) = {zero}
๐ป ๐ฉ is invertible! [ Note: ๐ฉ๐ฉ
๐ป is square, but NOT invertible!]
๐ป ๐ฉ
X๐ ๐ฉ
๐ป ๐ฉ = ๐ฐ ๐ฉ ๐ฅ๐๐๐ญ
X๐ (Dimension: ๐ฉ ๐ฅ๐๐๐ญ
X๐
๐ร๐
๐ฉ๐ร๐ = ๐ฐ๐ร๐ )
- ๐ = ๐ < ๐ Full row rank ( e.g. ๐ด = ๐/๐๐ก )
k = {zero}, no independent rows, but (๐ โ ๐) free variables
๐ป is invertible!
Define ๐จ๐จ
๐ป ๐จ๐จ
๐ป
X๐ = ๐ฐ so that ๐จ๐ร๐ ๐จ ๐ซ๐ข๐ ๐ก๐ญ
X๐
๐ร๐
= ๐ฐ๐ร๐
- One-sided inverse is NOT unique
Example : Integration and Differentiation
รผ We can change either the free column or the free row
รผ Moore-Penrose inverse (pseudo-inverse) ๐ด
W โ ๐ 7 ร 8
๐ด๐ต = ๐ผ, but ๐ต๐ด โ ๐ผ
Free column
Free row
H
H
The Moore-Penrose inverse is UNIQUE!
- Generally defined
- For full column (row) rank matrices, it reduces
to the particular left (right) inverse above
