Courant Institute, NYU
HONORS NUMERICAL ANALYSIS (MATH 396)
Spring 2023
Instructor
Mike O'Neil (oneil@cims.nyu.edu)

Office hours
Monday & Thursday, 11:00am - 12:00pm
(or by appointment)

Description
“Numerical analysis is the study of algorithms for the problems of continuous mathematics” - L. N. Trefethen, 1992. This course will cover the analysis of numerical algorithms which are ubiquitously used to solve problems throughout mathematics, physics, engineering, finance, and the life sciences. In particular, we will analyze algorithms for solving nonlinear equations; optimization; finding eigenvalues/eigenvectors of matrices; computing matrix factorizations and performing linear regressions; function interpolation, approximation, and integration; basic signal processing using the Fast Fourier Transform; Monte Carlo simulation. View the syllabus for more detailed information.

Lecture
Monday & Wednesday 2:00pm - 3:15pm, CIWW 201

Recitation
Alexandre Milewski (am10315@nyu.edu)
Friday 11:00am - 12:15pm, 194 Mercer, 307
Office hours: Wednesday 4:30pm - 5:30pm, CIWW 918
Materials
The following textbooks are recommended for reference material throughout the course:

- Overton, Numerical Computing with IEEE Floating Point Arithmetic, SIAM, 2001
- Driscoll and Braun, Fundamentals of Numerical Computation, SIAM, 2017
- Trefethen and Bau, Numerical Linear Algebra, SIAM, 1997
- Suli and Mayers, An Introduction to Numerical Analysis, Cambridge, 2003

Grading
The overall course grade will be determined from a final numerical weighted average. The following breakdown will be used to compute an overall numerical grade:
- 30% Homework (6 assignments, 5% each)
- 30% Midterm
- 40% Final exam (cumulative)

Announcements
None.
Schedule
Below is an updated list of discussion topics along with any documents that were distributed, notes, or relevant code.
Date Topics Materials
Jan 23 Overview, Julia Notes
Julia: A Fresh Approach to Numerical Computing
Jan 25 IEEE floating-point arithmetic Notes
macheps.jl
Driscoll & Braun, Floating-point numbers
Overton, 2001
Jan 30 Bisection, secant, and Newton's method Suli & Mayers: 1.4 - 1.6
Notes
Feb 1 Convergence of Newton's method,
rates of convergence, nonlinear systems
Suli & Mayers: 1.4, 1.7
Notes
Feb 6 Vector and matrix norms, conditioning Suli & Mayers: 2.7
Driscoll & Braun: 2.7
Driscoll & Braun: 2.8
Notes
Feb 9 Multivariate Newton Driscoll & Braun: 4.5
Notes
Feb 13 Optimization Driscoll & Braun: 4.6
Notes
Feb 15 Gaussian elimination, LU
Operation counts
Driscoll & Braun: 2.2
Driscoll & Braun: 2.3
Driscoll & Braun: 2.4
Driscoll & Braun: 2.5
Trefethen & Bau: Lectures 20-23
Notes
Feb 20 NO CLASS, PRESIDENTS' DAY
Feb 22 Cholesky, pivoted LU,
Backward stability, Gram-Schmidt
Driscoll & Braun: 2.6
Driscoll & Braun: 2.8
Driscoll & Braun: 2.9
Driscoll & Braun: 3.3
Trefethen & Bau: Lectures 20-23, 7-8
Notes
Feb 27 Modified Gram-Schmidt,
Householder reflections
Trefethen & Bau: Lectures 6-11
Notes
Mar 1 Householder reflections, linear regression,
singular value decomposition
Trefethen & Bau: Lectures 10-11,4-5
Notes
Mar 6 Power method, with shift Suli & Mayers: 5.4, 5.8
Notes
Mar 8 MIDTERM
Mar 13 NO CLASS - SPRING BREAK
Mar 15 NO CLASS - SPRING BREAK
Mar 20 Inverse power method with shift,
Jacobi's method
Suli & Mayers: 5.8, 5.3
Trefethen & Bau: Lectures 27, 30
Notes
Mar 22 Convergence of Jacobi's method,
QR algorithm
Suli & Mayers: 5.3, 5.5-5.7.2
QR Algorithm
Trefethen & Bau: Lectures 28-29
Notes
Mar 27 Numerically computing the SVD
Lagrange interpolation
Trefethen & Bau: Lectures 4, 31
Suli & Mayers: 6.1-6.2
Notes
Mar 29 Barycentric Lagrange interpolation,
minimax approximation
Suli & Mayers: 6.1-6.3, 8
Notes 1 Notes 2
Apr 3 Chebyshev interpolation, 2-norm approximation Suli & Mayers: Ch 8-9
Notes 1 Notes 2
Apr 5 Orthogonal polynomials,
trapezoidal rule
Suli & Mayers: Ch 9
Notes 1 Notes 2
Apr 10 Composite trapezoidal rule, Euler-MacLaurin,
Clenshaw-Curtis quadrature
Suli & Mayers: Ch 7
Notes
Apr 12 Richardson extrapolation, Gaussian-Quadrature Suli & Mayers: Ch 7.7, 10
Notes
Apr 17 Intro to Fourier series
Discrete Fourier analysis
Briggs & Henson: Ch 1, 2.1-2.5
Apr 19 NO CLASS
Apr 24 Fast Fourier Transform Briggs & Henson: Ch 10
Apr 26 Spectral differentiation and integration,
Clenshaw-Curtis quadrature (revisited)
FFT differentiation
Clenshaw-Curtis
May 1 Digital signal processing, convolutions Briggs & Henson: Ch 7.2
May 3 Iterative methods for linear systems: GMRES Trefethen & Bau: Lectures 32-35
May 8 Iterative methods for linear systems: GMRES
Review of course
Trefethen & Bau: Lectures 32-35
May 11 Final Exam 2:00pm - 3:50pm CIWW, Room 201