New THOTH website available here

DeepFlow

Philippe Weinzaepfel    Jerome Revaud    Zaid Harchaoui    Cordelia Schmid







Code

Please note that our code is mentioned only for scientific or personal use.

If you have any question, please contact:

Code for DeepMatching

See the dedicated webpage.

Source code for DeepFlow2

Once you have downloaded DeepMatching code, you can download DeepFlow2 code here for Linux or Windows (Visual C++, credits to Mitchel Benovoy for porting it) (NEW).
This will require to give the matches returned by DeepMatching as input matches.
DeepFlow2 is an improvement of DeepFlow that takes benefit of SSE instructions to speed up the computation. In addition, the code now incorporates a non-local weight for the smoothness term and the computation of matches' weights.
The code is released under the GNU General Public License.

Compilation

Simply type make and the code must compile.
The program was only tested under a 64-bit Linux distribution.
A static version is also provided in case of the compilation failure.
We do not give any support for other OS or compilation issues.

Usage

The general command to use DeepFlow2 is:

./deepflow2 <im1> <im2> <filename.flo> [options]
For a list of the available options, you can type:
./deepflow2 -h
The option -match allows to give input matches, either from an input file or from stdin.
In particular, results from the paper use the results from DeepMatching.

Example using DeepMatching

To compute DeepFlow2 including the DeepMatching, you need to download the Deep Matching code on the dedicated webpage. The matches output by DeepMatching can directly be piped to DeepFlow2 like in the following example:

<deepmatching_path>/deepmatching sintel1.png sintel2.png | ./deepflow2 sintel1.png sintel2.png sintel.flo -match -sintel
It creates a .flo file in a standard format. For instance, code for reading and displaying such format is available with the Middlebury dataset. For small size images, you can add the option "-downscale 0" when extracting DeepMatching for a better accuracy.

Warning: without matching

If you don't use any input match (i.e. you don't specify any -match argument), the method will be limited to a standard variational method without matching integration and thus will lose accuracy, specially for large displacements. This is the case in the opencv implementation.

History

Matlab wrapper

To compile the wrapper, first compile the code using make, then open matlab and compile the mex-file:

mex deepflow2_matlab.cpp image.o io.o opticalflow.o opticalflow_aux.o solver.o -ljpeg -lpng -lm -output deepflow2
The wrapper has been tested on Matlab14b under Linux-64 bit. We do not provide any support for compilation issue.

Usage with DeepMatching (assuming that DeepMatching has been added to the path with the matlab wrapper compiled; note that matlab must be opened with atlas preloaded for DeepMatching computation: LD_PRELOAD=/usr/lib64/atlas/libtatlas.so.3 matlab):

>> im1 = single(imread('sintel1.png')); >> im2 = single(imread('sintel2.png')); >> matches = deepmatching(im1, im2); >> flow = deepflow2(im1, im2, matches, '-sintel');
Calling deepflow2() will show the list of available options.

Python wrapper

The python wrapper requires numpy and (and swig if you modify the code and recompile it). To compile it, type make python. The wrapper has been tested under Linux-64 bit only. We do not provide any support for compilation issue.

Usage with DeepMatching (assuming that DeepMatching has been added to the pythonpath with the python wrapper compiled):

>> from deepmatching import deepmatching >> from deepflow2 import deepflow2 >> import numpy >> from PIL import Image >> im1 = numpy.array(Image.open('sintel1.png')) >> im2 = numpy.array(Image.open('sintel2.png')) >> matches = deepmatching(im1, im2) >> flow = deepflow2(im1, im2, matches, '-sintel')
Calling deepflow2() will show an the list of available options.

Citation

If you use our code, please cite our paper: 

@inproceedings{weinzaepfel:hal-00873592,
  AUTHOR = {Weinzaepfel, Philippe and Revaud, Jerome and Harchaoui, Zaid and Schmid, Cordelia},
  TITLE = {{DeepFlow: Large displacement optical flow with deep matching}},
  BOOKTITLE = {{IEEE Intenational Conference on Computer Vision (ICCV)}},
  YEAR = {2013},
  MONTH = Dec,
  ADDRESS = {Sydney, Australia},
  URL = {http://hal.inria.fr/hal-00873592}
}

Articles

ICCV 2013

DeepFlow: Large displacement optical flow with deep matching
Philippe Weinzaepfel, Jerome Revaud, Zaid Harchaoui and Cordelia Schmid,
Proc. ICCV‘13, December, 2013.   pdf    slides: ppt

IJCV 2016

DeepMatching: Hierarchical Deformable Dense Matching
Jerome Revaud, Philippe Weinzaepfel, Zaid Harchaoui and Cordelia Schmid,
IJCV 2016.   pdf