Computational Statistics

Entry level graduate class on computational statistics

• Optimization algorithms
• Models: likelihood, Bayesian posterior, mixture models
• Gradient descent, conditioning and pre-conditioning
• Newton and quasi-Newton (BFGS) methods, affine invariance
• L¹ regulatization, lasso regression, sparse fitting
• Alternating directions, EM (expectation maximization)
• Large datasets, stochastic gradient descent, etc.
• Monte Carlo methods
• (Pseudo) random number generators
• Direct samplers, mappings, rejection
• Verification, error bars, density estimation
• MCMC
  • Review of Markov chains
  • Detailed balance, Metropolis Hastings
  • Effective sample size, estimating the auto-correlation time
  • Modern samplers: Hamiltonian, ensemble, piecewise deterministic
  • Bayesian posterior: visualization
• Variance reduction: control variates, stratefied sampling
• Estimating integrals, evidence and model selection
• Filtering and prediction
• Uncertainty propagation
• Kalman filtering
• Particle filtering, extended Kalman filter
• Linear and non-linear feature selection
• Principal component analysis (PCA) and the SVD
• Non-linear features, kernel methods
• Deep non-linear features, neural nets

Prerequisites: Multivarite calculus (partial derivatives, multiple integrals), linear algebra through eigenvalues (real or complex) and eigenvectors. Upper level undergrad probability including multi-variate densities, conditional and marginal densities, commonly used distributions and the central limit theorem. Ability to code in Python or similar language. Helpful: experience with numerical computing, AI assisted coding.

Learning outcomes:

• Understand general issues of scientific computing, including inaccurate computer arithmetic, conditioning, work as a function of problem size, memory and data flow, etc.
• Understand the basic statistical problems and issues, including parameter estimation, hypothesis testing, model selection and undertainty quantification.
• Be able to formulate probabilistic models that allow for development and understanding of statistical methods for specific applications.
• Be able to code and use software tools to perform statistical computations, including code validation and reliability checks.
• Understand the pros and cons of frequentist and Bayesian approaches and select appropriate approaches for specific applications.
• Be able to assess the reliability of statistical methods using simulation, Monte Carlo, and analytic techniques (where available)
• Appreciate pitfalls and special properties of high dimensional, many parameter estimation applications and large datasets.
• Learn the material in the course outline above.

Textbook: Computational Statistics, by Geof Givens and Jennifer Hoeting

Assignments, exams, grading: The following is tentative. Depending on class size and motivation, there may be an oral component to the grade or a final project. The final grade will be based on weekly homework assignments (worth 15% of the grade), a midterm exam (25%) and a final exam (60%). In violation of New York State rules, there is no set grading scale. Those who achieve a majority of the goals listed above will receive A or A-. Grades B+ and B are for students who achieve a significant fraction of the goals. Grade B is not a good grade. Grades below B are "earned".

Communication: Please use the Brightspace site for content and homework communications. This way everyone sees and benefits from questions and answers, and there can be class discussion. Email the instructor for issues that do not involve others such as scheduling appointments, homework extensions, advice, etc.

Academic integrity:

• The NYU academic integrity policy applies and will be in force.
• Skills from this class are more important than the grade. Faking it might improve your grade, but it will hurt you in job interviews, future classes and research.
• Collabortion on assignments is encouraged, but you must write the solutions individually in your own words.
• It is forbidden to allow another student to copy your solutions.
• Students may not plagairize solutions from other students or sources such as books or web sites.
• AI policy: students may use AI tools for coding (Claude code, etc.) but not to create homework solutions. Warning: Do not trust AI generated facts or solutions that you do not understand. As of 2026, AI generated math content is often wrong.
• Please report (privately or even anonomously) any academic integrity violations you become aware of.
• Violation of these policies may result in grade lowering or more serious penalties, depending on severity and following math department guidelines.

Policies and information for all NYU classes: New York University is committed to providing equal educational opportunity and participation for students of all abilities. We work with NYU students to determine appropriate and reasonable accommodations that support equal access to a world-class education. Students requesting academic accommodations are advised to reach out to the Moses Center for Student Accessibility as early as possible in the semester for assistance.
Moses Center for Accessibility and Inclusive Culture
Telephone: 212-998-4980
Website: https://www.nyu.edu/life/community-flourishing/centers-and-communities/accessibility.html
Email: mosescenter@nyu.edu
Student Wellness: In a large, complex community like NYU, it's vital to reach out to others, particularly those who are isolated or engaged in self-destructive activities. Student wellness is the responsibility of all of us. The NYU Wellness Exchange is the constellation of NYU’s programs and services designed to address the overall health and mental health needs of its students. Students can access this service 24 hours a day, seven days a week. Contact information:
wellness.exchange@nyu.edu
Wellness Exchange hotline: 212-443-9999
NYU Counseling Service: 212-998-4780: appointments for single sessions, short-term, or group counseling.