



Study with the several resources on Docsity
Earn points by helping other students or get them with a premium plan
Prepare for your exams
Study with the several resources on Docsity
Earn points to download
Earn points by helping other students or get them with a premium plan
Community
Ask the community for help and clear up your study doubts
Discover the best universities in your country according to Docsity users
Free resources
Download our free guides on studying techniques, anxiety management strategies, and thesis advice from Docsity tutors
Material Type: Notes; Professor: Blomgren; Class: NUMERICAL MATRIX ANALYSIS; Subject: Mathematics; University: San Diego State University; Term: Spring 2010;
Typology: Study notes
1 / 6
This page cannot be seen from the preview
Don't miss anything!
The Big PictureFundamentalsFundamentals, ctd.
SVD, QR, Least Squares, Conditioning and Stability
Peter Blomgren, 〈blomgren.peter@gmail.com
Department of Mathematics and Statistics
Dynamical Systems Group Computational Sciences Research Center^ San Diego State UniversitySan Diego, CA 92182-7720^ http://terminus.sdsu.edu/
Spring 2010
Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
— (1/23)
The Big PictureFundamentalsFundamentals, ctd.
Outline^1
The Big Picture
Model Problem and Attacks; Analysis Building Blocks; Tools 2 Fundamentals
Basic Linear Algebra The SVD; Projections QR-Factorization 3 Fundamentals, ctd.
Linear Least Squares Conditioning, Stability and Accuracy Error Analysis & Stability Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
The Big PictureFundamentalsFundamentals, ctd.
Model Problem and Attacks; AnalysisBuilding Blocks; Tools
The Big Picture: Model Problem and Attacks
The Linear Least Squares Problem^ min^ ˜x∈C
‖An ˜x^ −
˜ b‖^2
m×n
,^ m
≥^ n
,^ rank
n
Attacks
Normal Equations
QR-Factorization
The SVD
Methods
Brute Force
Gram-Schmidt Orthogonalization
“magic”
Householder Triangularization
Modes Explicit
Q Implicit
∗˜ Qb
Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
— (3/23)
The Big PictureFundamentalsFundamentals, ctd.
Model Problem and Attacks; AnalysisBuilding Blocks; Tools
The Big Picture: Analysis
Conditioning
The inherent difficulty of the mathematical problemSensitivity to perturbations; quantified by the
condition number
,^ κ
Stability
The robustness of the algorithm Backward Stability
˜f^ (˜x) =
f^ (˜˜x),^
˜‖˜x^ −^ ˜ x‖=^ ‖˜x‖ O(ǫmach
)
Stability
˜ ‖f^ (˜x)^ −^ f^ (˜˜x )‖ ‖f^ (˜˜x)‖
,^ ˜‖˜x^ −^ ˜x ‖=^ O ‖˜x‖
(ǫmach
)
Accuracy
For a backward stable algorithm, the accuracy is
˜ ‖f^ (˜x) −^ f^ (
˜x)‖ ‖f^ (˜x)
‖^
=^ O(
κ(˜x)ǫ
)mach
Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
The Big PictureFundamentalsFundamentals, ctd.
Model Problem and Attacks; AnalysisBuilding Blocks; Tools
The Big Picture: Building Blocks / Tools
The SVD
Here used primarily (so far) for matrix understanding, expression of the condition numberof a matrix, simplification of proofs;
“every matrix is diagonal.”^ Projectors
(^2) P= P.^
Orthogonal if
∗^ P =^ P
.^ Can be formed using a orthogonal (
P^ =^
∗QQ ), or
non-orthogonal (
P^ =^
∗A(A −^1 A) ∗A) basis.^ Floating Point
A source of
unavoidable
errors in representation of numerical values, and computations.
Norms
Matrix and vector norms give us the fundamental measurements of size and distance inour vector spaces.^ Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
— (5/23)
The Big PictureFundamentalsFundamentals, ctd.
Basic Linear AlgebraThe SVD; ProjectionsQR-Factorization
Basic Linear Algebra, etc.
Always a Good Idea to Review
-^ Fundamental
matrix/vector
operations,
orthogonality,
or-
thonormality, inner products, the angle between two vectors,Hermitian transpose, linear independence, basis for a space,unitary matrices. • Vector and matrix norms, especially the
, and 2
norms, also the Frobenius norm of a matrix; 2-norm∞ invariance under multiplication by a unitary matrix. • Vector- and matrix-norm inequalities; Cauchy-Bunyakovsky-Schwarz. • The SVD as a tool for simplifying analysis and understandingof matrix properties. Geometric understanding. Unlikely
to show up as
explicit
questions on the midterm.
Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
The Big PictureFundamentalsFundamentals, ctd.
Basic Linear AlgebraThe SVD; ProjectionsQR-Factorization
The SVD
-^ (For now) a theoretical tool. •^ The full and reduced SVD. •^ Expressing
range
(A) and
null
(A) in terms of the components
of the SVD. • The singular values
rank
κ(A
‖, and^2
Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
— (7/23)
The Big PictureFundamentalsFundamentals, ctd.
Basic Linear AlgebraThe SVD; ProjectionsQR-Factorization
Projectors^ Definition (Projector)^ A
projector
is a square matrix
P^ that satisfies^2 P=
An^ orthogonal projector
is a projector that projects onto a
subspace
along a space
S, where^2
and
are orthogonal;
∗
v range(P) Pv
Peter Blomgren,
〈blomgren.peter@gmail.com
〉^
SVD, QR, LSQ, Conditioning and Stability
The Big PictureFundamentalsFundamentals, ctd.
Basic Linear AlgebraThe SVD; ProjectionsQR-Factorization
The Reduced and Full QR-Factorization^ As for the SVD, we can extend the QR-factorization by “fleshingout”
̂ Q^ with an additional (
m^ −
n) orthonormal columns, and
zero-padding
̂ R^ with an additional (
m^ −
n) rows of zeros: