Course information
Statistical learning is about the construction and study of systems that can automatically learn from data. With the emergence of massive datasets commonly encountered today, the need for powerful machine learning is of acute importance. Examples of successful applications include effective web search, anti-spam software, computer vision, robotics, practical speech recognition, and a deeper understanding of the human genome. This course gives an introduction to this exciting field, with a strong focus on kernels methods as a versatile tool to represent data, and recent convolutional and recurrent neural network models for visual recognition and sequence modeling.
Evaluation
- homework (1/2) + project (1/2)
Course outline
Introduction
- Motivating example applications
- Linear classification models
Deep learning models
- Convolutional neural networks
- Recurrent neural networks
Kernel Methods
- Theory of RKHS and kernels
- Supervised learning with kernels
- Unsupervised learning with kernels
- Kernels for structured data
- Kernels for generative models
Calendar
Classes take place from 9:45 to 12:45 on the following dates| Date | Lecturer | Room | Topic |
|---|---|---|---|
| 30/11/2017 | JM | H203 | Introduction [slides], kernels and RKHS [slides 1-52] |
| 7/12/2017 | JV | H203 | Fisher kernel [slides] + Intro neural nets [slides] |
| 14/12/17 | JM | H203 | Supervised and unsupervised learning with kernels [slides 68-91, 110-113, 128-135, 141-164, 170-183] |
| 21/12/17 | JV | D109 | Convolutional nets [slides] + Recurrent nets [slides] |
| 11/1/18 | JM | D109 | Kernels for sequence and graph data [slides 313-351, 395--407, 421--428], large-scale kernel learning [slides 532--535, 552--574], deep kernel learning [slides 577--585, 594--610] |
| 18/1/2018 | JV | D109 | Recurrent networks [slides] + Generative networks [slides] |
Homeworks
There will be two homeworks given during the course. Together they count for 50% of the grade. It can be done by groups of 2 students, and should be sent by e-mail (a Pdf file in LateX with the given template) to julien.mairal@m4x.org. A Latex template is available here. Note that you cannot work twice with the same person.- Homework 1: available here, to be handed in on December 22, 2017.
- Homework 2: available here, to be handed in on January 18, 2018.
- Data challenge: available; Follow the link given in class to register to the data challenge.
Projects
The project consists of experimenting with a learning approach of choice to solve a given prediction problem. A small (2-page max) written report has to be submitted to describe what you did, and results obtained. Code should also be submitted, as well as results.- Projects are due on February 15, 2018, on the Kaggle website.
- Reports are to be handed in by email two days later to dexiong.chen@inria.fr
- Projects can be done alone, or in groups of two people, but you cannot do your homework and the data challenge with the same person.
- Use your family names for the team names. Ex: Team verbeek_mairal.
- Any questions can be oriented to dexiong.chen@inria.fr
Reading material
Machine Learning and Statistics
- Vapnik, The nature of statistical learning theory. Springer
- Hastie, Tibshirani, Friedman, The elements of statistical learning. (free online)
- J Shawe-Taylor, N Cristianini. Kernel methods for pattern analysis. 2004.