Teaching: 2016, Fall (École Polytechnique):

Functional Data Analysis (MAP579):

• Goal:
  • The goal of the course is to enrich the participants with practical tools to be able to process and analyse data of functional nature, typically arising in climate research (for example, temperature curves, wind speed, and other time-series data). During the semester we will learn how to denoise functions (smoothing), align sample curves (time warping; shift registration, landmark registration, continuous registration), discover dominant patterns present in the data (dimensionality reduction), represent unequally spaced observations (functional representation), and visualize the obtained results. The course will also provide the possibility to work on hourly, atmospheric observations provided by the SIRTA-ReOBS project. The methods will be implemented in Matlab.
  • What you will learn: denoise experimental data, find recurrent patterns within the data, work with time series, and see applications of machine learning to process experimental data.
• Prerequisites:
  • The course requires a basic knowledge of probability theory (random variable, density function, mean, variance) at the level of 'A First Course in Probability by Sheldon Ross', calculus (function, derivative, integrals) at the level of 'Calculus by James Stewart', linear algebra (vector, matrix) at the level of 'Introduction to Linear Algebra by Serge Lang'.

---

• Lecture 1 (Oct. 4):
  • Keywords: smoothing by least squares, basis function technique, linear smoother, localized least squares (kernel smoothing), Nadaraya-Watson estimator, Gasser-Müller estimator, localized basis function estimator, local polynomial smoothing.
  • Slides: maths, quick summary of Matlab.
  • Data: weather data.
  • Covered: Chapter 1-4 of [1].
• Lecture 2 (Oct. 11):
  • Keywords (maths): smoothing with roughness penalty (regularization approach), harmonic acceleration operator, spline smoothing, B-spline basis, degree of freedom, quadrature rules (trapezoid rule, Simpson's rule), (generalized) cross-validation, bi-resolution analysis.
  • Keywords (FDA toolbox): basis object, create/evaluate/plot basis systems.
  • Slides: maths, FDA package: basics.
  • Covered: Chapter 5 of [1], Chapter 1-3 of [2].
• Lecture 3 (Oct. 18):
  • Keywords (maths): smoothing with constraints (positivity, monotonicity, probability density function), maximum-likelihood estimation, curve registration, amplitude/phase variability, shift registration, Procrustes method, feature or landmark registration, time-warping function, continuous registration, modified Newton-Raphson method.
  • Keywords (FDA toolbox): fd, fdnames, Lfd, fdPar objects.
  • Slides: maths, FDA package: smoothing.
  • Covered: Chapter 6-7 of [1], Chapter 4-5 of [2].
• Lecture 4 (Nov. 8):
  • Keywords: dimensionality reduction, principal component analysis (Karhunen-Loeve transformation, Hoteling transformation), constrained optimization (Lagrange multipliers).
  • Slides: maths, PCA tasks.
  • Covered: 'R^d part' of Chapter 8 in [1], and [3].
• Lecture 5 (Nov. 15):
  • Keywords: functional PCA, covariance operator, (generalized) eigenvalue problems.
  • Slides: maths, FDA package: curve registration.
  • Code: patched smooth_monotone.m, registration (growth).
  • Covered: 'functional part' of Chapter 8 in [1].
• Lecture 6 (Nov. 22):
  • Keywords: regularized functional PCA.
  • Slides: maths + Matlab.
  • Code: image compression demo (PCA), SIRTA public data.
  • Covered: Chapter 9 in [1], 'functional PCA part' of Chapter 7 in [2].
• Lecture 7 (Dec. 13):
  • Keywords: inner product space, norm, Hilbert space, CBS inequality, kernel, reproducing property, RKHS, kernel ridge regression, representer theorem, covariance operator, kernel PCA.
  • Slides.
  • References: [4-6].
• References:
  • [1] J.O. Ramsay, B.W. Silverman. Functional Data Analysis. Springer, 2005. [link]
  • [2] J.O. Ramsay, Giles Hooker, Spencer Graves. Functional Data Analysis with R and Matlab. Springer, 2009. [link]
  • [3] Cosma Shalizi. Notes on 'Principal Components Analysis', 2016. [link]
  • [4] Bernhard Schölkopf, Alex Smola, Klaus-Robert Müller. Kernel Principal Component Analysis, pages 583-588, ICANN-1997.
  • [5] Sebastian Mika, Bernhard Schölkopf, Alex Smola, Klaus-Robert Müller, Matthias Scholz, Gunnar Rätsch. Kernel PCA and De-Noising in Feature Spaces, pages 536-542, NIPS-2009.
  • [6] Ingo Steinwart, Andreas Christmann. Support Vector Machines, 2008.
• Code/dataset (external):
  • FDA toolbox,
  • fastICA (specifically PCA), natural image dataset.