Course information

Machine Learning is about the construction and study of systems that can automatically learn from data. With the emergence of massive datasets commonly encountered today, the need for powerful machine learning is of acute importance. Examples of successful applications include effective web search, anti-spam software, computer vision, robotics, practical speech recognition, and a deeper understanding of the human genome. In this course, we will give an introduction to this exciting field. We will focus on supervised learning, such as classification and ranking, and unsupervised learning problems, such as clustering and dimension reduction. We will study classical algorithms, and introduce tools to measure their performance, as well as their computational complexity.

Related courses

  • CR01: Using Randomness in Science
  • CR07 Algorithms for Molecular Biology
  • CR08: Combinatorial Scientific Computing

Evaluation

  • homeworks 33%
  • project 67%

Course outline

Introduction

  • Empirical risk minimization
  • Risk convexification and regularization
  • Bias-variance trade-off, and risk bounds

Supervised learning

  • Ridge regression
  • Logistic regression
  • Perceptron and neural networks
  • Support vector machines
  • Other methods: nearest-neighbors, kernel methods, etc.

Unsupervised learning

  • Principal component analysis
  • Data clustering
  • Other methods: canonical correlation analysis, sparse coding, etc.

Reading material

Machine Learning and Statistics

  • Vapnik, The nature of statistical learning theory. Springer
  • Hastie, Tibshirani, Friedman, The elements of statistical learning. (free online)
  • Devroye, Gyorfi, Lugosi, A probabilistic theory of pattern recognition. Springer
  • Dubashi, Panconesi, Concentration of measure for analysis of randomized algorithms, Cambridge University Press
  • J Shawe-Taylor, N Cristianini. Kernel methods for pattern analysis. 2004.

Optimization

  • S. Boyd and L. Vandenberghe. Convex Optimization. 2004. (free online)
  • D. Bertsekas. Nonlinear Programming. 2003.

Calendar

Date Lecturer Topic Scribes
20/09 ZH Introduction + Perceptron.
notes
Sergio Peignier
Semen Marchuk
04/10 ZH Kernel Methods + Stochastic Learning.
notes
Sébastien Maulat
Fasi Massimiliano
18/10 JS Stochastic Learning + Risk bounds.
notes
Bertrand Simon
Antoine Plet
08/11 JM Supervised Learning - Methodology.
notes
Jordan Frecon
Merve Ünlü
22/11 JM Examples of Kernels - Unsupervised Learning.
notes
Rémi De Joannis de Verclos
Karthik Srikanta
06/12 JS Random projections - Nearest Neighbors.
notes
Tic
Tac

Scribe notes

For each course, a duo of students commit to turn their notes into latex format. A cool package, due to S. Maulat and A. Plet, for making nice notes, can be found here.

Homeworks

During each course, a few exercises are given. They should be returned to the lecturers during the last course on December 6th. (no need to use LateX here).

Projects

The project consists of implementing an article, doing some experiments, and writing a small report (less than 10 pages). It is also possible to study a theoretical paper instead of implementing a method. All reports should be written in LateX, and a pdf should be sent to the lecturers before January 5th.
Project Student(s) Coach
Supervised classification of text documents.
material
Rémi De Joannis de Verclos
Sébastien Maulat.
JM
Machine learning for games.
material
Massimiliano Fasi. JS
Face recognition.
material
Semen Marchuk
Bertrand Simon.
JM
Predicting Molecular Activity with Graph Kernels.
material
Sergio Peignier. JM
Speaker Recognition.
material
Antoine Plet
Thomas Sibut-Pinote.
ZH
Active Learning.
material
material
Guillaume Sergent. ZH, JS
Spectral Clustering Theory.
material
Karthik Srikanta. ZH
Supervised classification of Flickr images.
material
Merve Ünlü. JS

Jobs / Internships Opportunities

We have lots of intern/PhD opportunities in machine learning and its applications, such as image processing, bioinformatics and computer vision. Please contact the lecturers for more information. Some more detailed topics are listed below:
  • Large-Scale Machine Learning for Statistical Genetics with Julien Mairal and Michael Blum (Grenoble)
  • Machine Learning for web-link prediction, with applications to Knowledge Graphs, with Z. Harchaoui, M. Amini, and A. Juditsky (Grenoble)
  • Statistical aggregation : greedy algorithms vs MCMC, with Joseph Salmon (Paris)