Goal

The 1st edition of the "Khronos Days" summer school is focused on

High-Dimensional Learning and Optimization

The amount of digital data is currently increasing at an exponential rate. This data deluge is a wonderful opportunity to extract previously unknown information from the data, and therefore a major leverage for scientific advances. Yet such big data raises several major challenges related to high-dimensional data and models, tackled by recent research in mathematical optimization and statistical learning. The goal of this summer school is to provide an overview of some of the latest advances in these fields. Each lecture is a tutorial, starting from the basic concepts and tools, and progressively moving further to the latest advances.

A particular emphasis will be given on covering the major theoretical results along with their proofs, as well as illustrations on real-world applications. Lectures will be given by experts in mathematical optimization and statistical learning. Some lectures will be complemented by practical sessions, where participants will obtain hands-on experience with the discussed material.

The summer school is a scientific event organized as part of the Khronos project of the Labex Persyval and the Gargantua project of the CNRS-Mastodons program.

Speakers and program

Registration and ECTS credits

If you did not register, please feel free to come by. You may register directly at the registration desk in front of the Amphi. door.

PhD attendees from Grenoble will get an official letter from the organizers that will allow them to earn ECTS credits and validate the courses as part of their "formation doctorale".

Schedule and location

The summer school will be held from June 10th to June 12th in Grenoble. Note that, on June 12th, the location changes from Amphi. Chabauty-Institut Fourier to Maison Jean Kuntzmann.
  • On June 10th and June 11th, the lectures will be given in Amphi. Chabauty, Institut Fourier.
  • On June 12th, the lectures will be given in Maison Jean Kuntzmann.
The Amphi. Chabauty is located here. The Maison Jean Kuntzmann is located here. The summer school schedule below may be subject to changes before June 10th 2014; please check the webpage before then.
  • Tuesday, June 10th, 2014. Amphi. Chabauty, Institut Fourier
    • 13:45. Welcome and Introduction
    • 14:00. Arnak Dalalyan. Recent advances on the risk bounds for the Lasso and related methods, Part I
    • 17:00. Coffee
  • Wednesday, June 11th, 2014. Amphi. Chabauty, Institut Fourier
    • First half-day
    • 09:00. Arnak Dalalyan. Recent advances on the risk bounds for the Lasso and related methods, Part II
    • 12:00. Lunch pause
    Second half-day
    • 14:00. Peter Richtarik. Randomized Gradient Methods for Big Data Optimization, Part I
    • 17:00. Coffee
  • Thursday, June 12th, 2014. Maison Jean Kuntzmann
    First half-day
    • 09:00. Peter Richtarik. Randomized Gradient Methods for Big Data Optimization, Part II
    • 12:00. Lunch pause
    Second half-day
    • 14:00. Vladimir Koltchinskii, Probabilistic Tools in Large Matrix Estimation
    • 17:00. Closing remarks and Coffee.

Organizers

Abstracts of the talks

  • Recent advances on the risk bounds for the Lasso and related methods, Arnak Dalalyan, 2*3hrs
    The Lasso and other methods based on ell-one penalization are widely used now in various applications for performing estimation, prediction, variables selection, etc. In these lectures, I will present an overview of the risk bounds for the Lasso available in statistical and machine learning literature, with an emphasis on bounds leading to fast rates under prediction loss. Most recent results will be presented with detailed proofs and a discussion of various assumptions on the dictionary (restricted eigenvalues, compatibility, weighted compatibility) under which fast rates are valid. I will also provide a simple counter-example showing the impossibility of getting fast rates when no condition on the dictionary is imposed. Further, I will present an approach allowing to analytically check the compatibility condition. The course is aimed at PhD students in disciplines such as mathematical optimization, machine learning, computer science, statistics, applied mathematics and engineering. However, it is also suitable to researchers in various quantitative disciplines interested in the topic.


  • Probabilistic Tools in Large Matrix Estimation, Vladimir Koltchinskii, 3hrs
    We will discuss several problems related to estimation of large matrices including low rank matrix recovery, trace regression and covariance estimation. The main focus will be on probabilistic tools needed to provide sharp nonasymptotic bounds on the estimation error in relevant norms in the spaces of matrices (such as the operator norm, the Hilbert--Schmidt norm, etc). These tools include probabilistic inequalities for sums of independent random matrices (such as, for instance, noncommutative Bernstein type inequalities) as well as more general concentration inequalities and moment bounds for empirical processes. The course is aimed at PhD students in disciplines such as mathematical optimization, machine learning, computer science, statistics, applied mathematics and engineering. However, it is also suitable to researchers in various quantitative disciplines interested in the topic.


  • Randomized Gradient Methods for Big Data Optimization, Peter Richtarik, 2*3hrs
    Many big data applications can be cast as optimization problems and solved by a suitable optimization algorithm. Due to the size of such problems, simple methods able to progress while investigating only a small portion of the data are more desirable and efficient. This short course presents a unified theory of a large class of randomized block coordinate descent methods, which have recently become extremely popular in areas such as machine learning, optimization and engineering due to their simplicity, versatility, scalability and ability to take advantage of sparsity in data. In the special case of trivial randomization, these methods become deterministic and include algorithms such as gradient descent, projected gradient descent, proximal gradient descent, iterative soft thresholding algorithm (ISTA), Nesterov's accelerated gradient method and FISTA. In the general randomized setting, we shall talk about serial, parallel, distributed, proximal and accelerated coordinate descent. The course offers a unified view where all these methods appear as special cases of a more general algorithm. The course is aimed at PhD students in disciplines such as mathematical optimization, machine learning, computer science, statistics, applied mathematics and engineering. However, it is also suitable to researchers in various quantitative disciplines interested in the topic.