New THOTH website available here

DeepMatching: Deep Convolutional Matching


Jerome Revaud         Philippe Weinzaepfel         Zaid Harchaoui         Cordelia Schmid






Description

DeepMatching is a matching algorithm developped by Jerome Revaud in 2013. Its purpose is to compute dense correspondences between two images. DeepMatching relies on a deep, multi-layer, convolutional architecture designed for matching images. It can handle non-rigid deformations and repetitive textures, and can therefore efficiently determine dense correspondences in the presence of significant changes between images.
We have evaluated the performance of DeepMatching, in comparison with state-of-the-art matching algorithms, on several datasets (see technical report below for details). DeepMatching outperforms the state-of-the-art algorithms in terms of accuracy and shows excellent results in particular for repetitive textures.
DeepMatching was recently used to improve the estimation of optical flow in several methods like DeepFlow and EpicFlow (joint work with Philippe Weinzaepfel, Zaid Harchaoui, Cordelia Schmid). Results on MPI-Sintel are currently state-of-the-art.

Download

Please note that our code is mentioned only for scientific or personal use.

The source code for DeepMatching can be found here:

Installation and tutorial

After downloading the code, unzip it in some folder and compile it with make. The compilation has been tested on Linux (Fedora 21) and Mac OS X (CPU version only). For Linux, we have included a pre-compiled executable (CPU version only).

Arguments and options

./deepmatching <img1_path> <img2_path> -nt <n_threads> 
To sum it up, deepmatching takes as arguments the paths towards two input images, and that's about it (no training required). Fine-tuning of the internal parameters is still possible, but unnecessary in practice as their impact is often unnoticeable. To get some help about other program arguments, execute:

./deepmatching --help

Matching visualization

In order to easily visualize the output correspondences, we provide a friendly GUI viz.py in which you can examine correspondences one by one by moving your mouse. For instance, to generate the figure shown on top of this page, simply pipe the output of deepmatching into viz.py:

./deepmatching climb1.png climb2.png -nt 0 | python viz.py climb1.png climb2.png
For it to work, you must have installed python 2.7, PIL (Python Image Library), numpy and matplotlib.

Citation

If you use our code, please cite our paper:

@inproceedings{weinzaepfel:hal-00873592,
  AUTHOR = {Weinzaepfel, Philippe and Revaud, Jerome and Harchaoui, Zaid and Schmid, Cordelia},
  TITLE = {{DeepFlow: Large displacement optical flow with deep matching}},
  BOOKTITLE = {{IEEE Intenational Conference on Computer Vision (ICCV)}},
  YEAR = {2013},
  MONTH = Dec,
  ADDRESS = {Sydney, Australia},
  URL = {http://hal.inria.fr/hal-00873592}
}

Articles

Extended paper (IJCV)

DeepMatching: Hierarchical Deformable Dense Matching
Jerome Revaud, Philippe Weinzaepfel, Zaid Harchaoui and Cordelia Schmid,
IJCV, 2016.   pdf

ICCV 2013 (Oral)

DeepFlow: Large displacement optical flow with deep matching
Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui and Cordelia Schmid,
Proc. ICCV‘13, December, 2013.   pdf    slides: ppt

CVPR 2015 (Oral)

EpicFlow: Edge-Preserving Interpolation of Correspondences for Optical Flow
Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui and Cordelia Schmid
CVPR 2015.   pdf