Course: 2021/2022

Computational and Applied Linear

(18768)

Requirements (Subjects that are assumed to be known)

Linear Algebra.
Calculus I.
It is advisable to have some basic knowledge of MATLAB.

The student will get familiar with the basic algorithms for solving the main four problems of numerical linear algebra (NLA), namely: (1) the solution of linear systems, (2) the solution of least squares problems, (3) the computation of eigenvalues and eigenvectors, and (4) the computation of the singular value decomposition (SVD). Also, he/she will acquire techniques and tools from NLA that can be useful either in his/her professional performance, in areas like data analysis and pattern recognition, or in a scientific career in the field of applied and computational mathematics. In particular, the student will learn and will be able to manage:
- The basic MATLAB commands in the context of the four main problems in NLA mentioned above.
- The fundamentals of numerical analysis (conditioning, stability, and computational complexity).
- The error analysis in numerical methods, in particular those appearing in NLA.
- The basic facts of the floating point arithmetic.
- The basic notions of matrix norms, together with their relevance in the numerical computations that involve the use of matrices.
- The tools and the theory underlying the algorithms that are currently employed for the solution of linear systems, both for matrices with small to moderate size (direct methods) and for large scale matrices (iterative methods).
- The tools and the theory underlying the algorithms that are currently employed for the computation of eigenvalues and eigenvectors, both for matrices with small to moderate size (direct methods) and for large scale matrices (iterative methods).
- The theory and tools in the computation of the SVD, as well as an approach to the basic algorithms for computing such decomposition.
- The basic theory and tools in the solution of least squares problems.
- Some of the applications of the SVD in both theoretical and applied frameworks, like the distance to the set of matrices with smaller rank or the image compression and the principal component analysis.
- Some of the standard applications of NLA in applied contexts, like data analysis or image recognition.
Competences associated with this subject:
In this subject, the student will make progress in achieving the following competences that are indicated in the ¿Memoria de verificación¿ of the Master:
Basic competences:
CB6: Having and understanding the knowledge that provides a basis or opportunity to be original in the development and/or application of ideas, often in a research context.
CB7: Students know how to apply their acquired knowledge and problem-solving skills in new or unfamiliar settings within broader (or multidisciplinary) contexts related to their field of study.
CB8: Que los estudiantes sean capaces de integrar conocimientos y enfrentarse a la complejidad de formular juicios a partir de una información que, siendo incompleta o limitada, incluya reflexiones sobre las responsabilidades sociales y éticas vinculadas a la aplicación de sus conocimientos y juicios. Students are able to integrate knowledge and to face the complexity of making judgments based on information that, being incomplete or limited, includes reflections on the social and ethical responsibilities linked to the application of their knowledge and judgments.
CB10: Students have the learning skills that will enable them to continue studying in a way that will be largely self-directed or autonomous.
General competences:
CG1: Collect and interpret data of a mathematical nature which can be applied to other domains of scientific knowledge.
CG2: Apply acquired knowledge and possess the ability to solve novel problems related with Mathematics.
CG4: Being able to generate new ideas which may imply an advance of knowledge for Mathematics
CG5: Being able to communicate conclusions in clear and precise way.
CG6: Being able to autonomously study and do research.
CG7: Being able to do team-work and manage available time.
Specific competences:
CE1: Understanding and properly using mathematical language.
CE2: Being able to formulate mathematical statements in various fields and set up proofs.
CE3: Being able to abstract structural properties differentiating them from more accidental ones.
CE4: Being able to solve mathematical problems, planning their solution in terms of the available tools and of additional time and resource limitations.
CE5: Being able to develop computer software which solves mathematical problems using the most suitable computational environment in each case.
CE6: Being able to design and implement more or less complex algorithms to solve real-life problems.
CE8: Being able to reflect on obtained results, formulating their domain of validity and/or applicability.
CE11: Being able to understand and apply advanced knowledge on numerical methods and computing to problems in science, technology, and society.

Description of contents: programme

1. Fundamentals on numerical analysis.
1.1 Introduction to floating point arithmetic.
1.2 Conditioning and stability.
1.3 Computational cost.
2. Vector and matrix norms.
2.1 Vector norms.
2.2 Induced matrix norms.
2.3 General matrix norms.
2.4 The spectral radius.
3. The Schur normal form.
3.1 Unitary matrices.
3.2 The Schur triangular form.
3.3 Some applications.
4. Direct methods for solving linear systems.
4.1 Gaussian elimination and LU factorization.
4.1.1 Gaussian elimination and LU without pivoting.
4.1.2 LU with pivoting: partial and total pivoting.
4.1.3 Stability analysis of the LU factorization. The growth factor.
4.2 Symmetric matrices: The Cholesky factorization.
4.3 QR factorization.
4.3.1 QR factorization and Gram-Schmidt.
4.3.2 QR factorization using Householder reflections.
5. Direct methods for computing eigenvalues and eigenvectors.
5.1 Reduction to Hessenberg and tridiagonal form.
5.2 The QR algorithm.
5.2.1 The power and inverse power method with shifts.
5.2.2 Subspace iteration.
5.2.3 The QR algorithm.
5.2.4 The QR algorithm in practice.
5.2.5 Algorithms for symmetric matrices.
5.3 Sensitivity of the eigenvalue problem.
6. The SVD: computation and some applications.
6.1 The SVD: existence and basic features.
6.2 Algorithms for computing the SVD.
6.3 Applications of the SVD in low-rank approximation problems.
6.3.1 Distance to singularity.
6.3.2 Image compression.
6.3.3 Principal Component Analysis.
6.3.4 Other applications.
7. Least squares problems.
7.1. Least squares problems: basic properties.
7.2 Solution of least squares problems. The pseudoinverse.
7.3 Normal equations vs QR for solving least squares problems.
7.4 Least squares problems using the SVD.
8. Iterative methods for solving linear systems and computing eigenvalues.
8.1 Krylov-type methods.
8.1.1 The Krylov methods.
8.1.2 The conjugate gradient method.
8.1.3 Other methods.
8.2 Lanczos and Arnoldi methods for computing the eigenvalues.
9. Further applications of Numerical Linear Algebra.
9.1 PageRank.
9.2 Matrix completion.
9.3Data Mining.

Learning activities and methodology

The course is conceived as a practical course from the computational point of view. All lectures (that will be carried out, presumably, in the computer laboratory) will include, at least, one practice with the MATLAB program. The participation in these practices will be fundamental in order to acquire the objectives of the course. The practices will be developed in small groups and the solutions will be provided to the professor before the end of the quarter.
The course includes a basic training in the MATLAB program, and the basic tools, codes, and basic programming skills will be provided to the students.
The notes of the course will be available for the students from the very beginning, with all the contents that will be addressed during the course. These notes will include a series of exercises that will allow the students to go deeper in the contents of the course, and/or will give them the opportunity to solve by themselves some of the open questions in the notes.
There will be some teaching ours, according to the official rules of the university.

Assessment System

- % end-of-term-examination 40
- % of continuous assessment (assigments, laboratory, practicals...) 60

Basic Bibliography

- Biswa N. Datta. Numerical Linear Algebra and Applications, 2nd ed. SIAM. 2010
- David S. Watkins. Fundamentals of Matrix Computations. John Wiley and Sons. 2002
- James W. Demmel. Applied Numerical Linear Algebra. SIAM. 1997
- Nicholas J. Higham. Accuracy and Stability of Numerical Algorithms. SIAM. 2002

Additional Bibliography

- Gene Golub, Charles Van Loan. Matrix Computations, 4th ed. The Johns Hopkins University Press. 2002
- George W. Stewart, Ji-Guang Sun. Matrix Perturbation Theory. Academic Press. 1990
- Gilbert Strang. Linear Algebra and Learning from Data. Wellesley Cambridge Press. 2019
- Ilse Ipsen. Numerical Matrix Analysis. SIAM. 2009
- Lars Eldén. Matrix Methods in Data Mining and Pattern Recognition. SIAM. 2007
- Michael L. Overton. Numerical Computing with IEEE Floating Point Arithmetic. SIAM. 2001
- Roger A. Horn, Charles R. Johnson. Matrix Analysis, 2nd ed. Cambridge University Press. 2013