Course information

Statistical learning is about the construction and study of systems that can automatically learn from data. With the emergence of massive datasets commonly encountered today, the need for powerful machine learning is of acute importance. Examples of successful applications include effective web search, anti-spam software, computer vision, robotics, practical speech recognition, and a deeper understanding of the human genome. This course gives an introduction to this exciting field, with a strong focus on kernels methods as a versatile tool to represent data, and recent convolutional and recurrent neural network models for visual recognition and sequence modeling.

Evaluation

  • exam (40%) + homework (30%) + project (30%)

Course outline

Introduction

  • Motivating example applications
  • Linear classification models

Deep learning models

  • Convolutional neural networks
  • Recurrent neural networks
  • Unsupervised deep learning

Kernel Methods

  • Theory of RKHS and kernels
  • Supervised learning with kernels
  • Unsupervised learning with kernels
  • Kernels for structured data
  • Kernels for generative models

Rattrapage - Second Session

The second chance exam will consists of a homework, which is available here. The homework was posted on April 19th, 10:20am. We expect them to be handed back on April 19th before 9pm by e-mail to julien.mairal@inria.fr. Any scanned copy would work.

Calendar

Date Time Lecturer Room Topic
29/11/2018 11:15 - 12:45 JV D109 Course introduction [slides] + Introduction Deep Learning [slides]
6/12/2018 11:15 - 12:45 JV D109 Intro DL + MLPs [ slides ]
13/12/2018 9:45 - 12:45 JM D109 kernels and RKHS [slides 1-61]
20/12/2018 9:45 - 12:45 JV D109 Convolutional networks [slides] + Recurrent networks [slides]
10/1/2019 9:45 - 12:45 JM D109 Kernel tricks - Kernel Ridge -Regression - SVMs [slides 62-96 and 133-160]
15/1/2019 9:45- 11:15 JV D109 Generative models [slides]
16/1/2019 17:00- 18:30 JV D109 Generative models [slides]
24/1/2019 9:45- 12:45 JM D109 kernel PCA, kernel K-means, string kernels [slides 165-171, 175-186, and 321-359]

Homeworks

There will be one homework given during the course. It counts for 30% of the grade. It can be done by groups of 2 students, and should be sent by e-mail (a Pdf file in LateX with the given template) to julien.mairal@m4x.org. A Latex template is available here.
  • Homework: available here since December 19th 2018, to be handed in on January 7th, 2019.
  • Exam, 2pm - 4pm, Ensimag A010, February 1, 2019, Authorized material: A single A4 sheet with own notes
  • List of students for the class here .

Projects

The project consists of experimenting with a learning approach of your choice to solve a given prediction problem. A small (2-page max) written report has to be submitted to describe what you did, and results obtained. Code should also be submitted, as well as results. The project counts for 30% of your grade.
  • The data and documentation are available here from December 20, 2018.
  • Projects are due on February 11, 2019 on the Kaggle website.
  • Code and reports are to be handed in by email to dexiong.chen@inria.fr on February 13th, 2019.
  • Projects can be done alone, or in groups of two people, but you cannot do your homework and the data challenge with the same person.
  • Use your family names for the team names. Ex: Team verbeek_mairal.
  • Any questions can be oriented to dexiong.chen@inria.fr

Reading material

Machine Learning and Statistics

  • Vapnik, The nature of statistical learning theory. Springer
  • Hastie, Tibshirani, Friedman, The elements of statistical learning. (free online)
  • J Shawe-Taylor, N Cristianini. Kernel methods for pattern analysis. 2004.

Jobs / Internships Opportunities

We have different intern/PhD opportunities in machine learning, image processing, bioinformatics and computer vision. It is best to discuss that matter early with us since the number of places is limited.