DeepMatching: Deep Convolutional Matching
Description
DeepMatching is a matching algorithm developped by Jerome Revaud in 2013.
Its purpose is to compute dense correspondences between two images. DeepMatching relies on a deep,
multi-layer, convolutional architecture designed for matching images.
It can handle non-rigid deformations and
repetitive textures, and can therefore efficiently determine dense
correspondences in the presence of significant changes between images.
We have evaluated the performance of DeepMatching, in comparison with state-of-the-art matching algorithms,
on several datasets (see technical report below for details).
DeepMatching outperforms the state-of-the-art algorithms in terms of accuracy and
shows excellent results in particular for repetitive textures.
DeepMatching was recently used to improve the estimation of optical flow in several methods like DeepFlow and EpicFlow (joint work with Philippe Weinzaepfel, Zaid Harchaoui, Cordelia Schmid). Results on MPI-Sintel are currently state-of-the-art.
Download
Please note that our code is mentioned only for scientific or personal use.
The source code for DeepMatching can be found here:CPU version:
- Version 1.0.2 (February 1st, 2015)
- Version 1.1: (May 10th, 2015)
- Added new mode for full scale and rotation invariance.
- Improved visualization (viz.py)
- Removed useless/suboptimal options (eg. -iccv_settings)
- Fixed a bug related to memory allocation for large images - Version 1.2: (May 20th, 2015)
- Added a new option "-ngh_rad" to restrict the matching to a local neighborhood, which enables much reduced memory usage and computations.
- Statically-compiled version is now fully multi-threaded with BLAS (but still much slower than if you compile it yourself).
- Few minor bugfix, code cleaning and updates. - Version 1.2.1: (September 8th, 2015)
- Now performing the maxpooling and subsampling steps jointly, which results in using 50% less memory than before. Also, it is a bit faster.
- Removed some useless/confusing options in the executable. - Version 1.2.2: (October 19th, 2015)
- Now include a Matlab and a Python wrapper!
- Windows version (Visual C++) now available (Credit to Mitchel Benovoy for porting it). GPU version:
- Version 1.0: (November 16th, 2015) NEW
- requires Python2.7, possibly SWIG (not mandatory), and caffe's dependencies (caffe is provided).
Installation and tutorial
After downloading the code, unzip it in some folder and compile it with make. The compilation has been tested on Linux (Fedora 21) and Mac OS X (CPU version only). For Linux, we have included a pre-compiled executable (CPU version only).
Arguments and options
./deepmatching <img1_path> <img2_path> -nt <n_threads>To sum it up, deepmatching takes as arguments the paths towards two input images, and that's about it (no training required). Fine-tuning of the internal parameters is still possible, but unnecessary in practice as their impact is often unnoticeable. To get some help about other program arguments, execute:
./deepmatching --help
Matching visualization
In order to easily visualize the output correspondences, we provide a friendly GUI viz.py in which you can examine correspondences one by one by moving your mouse. For instance, to generate the figure shown on top of this page, simply pipe the output of deepmatching into viz.py:
./deepmatching climb1.png climb2.png -nt 0 | python viz.py climb1.png climb2.pngFor it to work, you must have installed python 2.7, PIL (Python Image Library), numpy and matplotlib.
Citation
If you use our code, please cite our paper:
@inproceedings{weinzaepfel:hal-00873592,
AUTHOR = {Weinzaepfel, Philippe and Revaud, Jerome and Harchaoui, Zaid and Schmid, Cordelia},
TITLE = {{DeepFlow: Large displacement optical flow with deep matching}},
BOOKTITLE = {{IEEE Intenational Conference on Computer Vision (ICCV)}},
YEAR = {2013},
MONTH = Dec,
ADDRESS = {Sydney, Australia},
URL = {http://hal.inria.fr/hal-00873592}
}
Articles
Extended paper (IJCV)
ICCV 2013 (Oral)
slides: pptCVPR 2015 (Oral)