The most popular optical flow algorithms rely on optimizing the energy function that integrates a data term and a smoothness term. In contrast to this traditional framework, we derive a new objective function that couples optical flow estimation and image restoration. Our method is inspired by the recent successes of edge-aware constraints (EAC) in preserving edges in general gradient domain image filtering. By incorporating an EAC image fidelity term (IFT) in the conventional variational model, the new energy function can simultaneously estimate optical flow and restore images with preserved edges, in a bidirectional manner. For the energy minimization, we rewrite the EAC into gradient form and optimize the IFT with Euler–Lagrange equations. We can thus apply the image restoration by analytically solving a system of linear equations. Our EAC-combined IFT is easy to implement and can be seamlessly integrated into various optical flow functions suggested in literature. Extensive experiments on public optical flow benchmarks demonstrate that our method outperforms the current state-of-the-art in optical flow estimation and image restoration.

+More

image restoration eaccombined ift gradient form smoothness term eac image fidelity term stateoftheart preserving edges energy minimization gradient domain image filtering framework eulerlagrange method edgeaware constraints restore images linear equations public optical flow benchmarks variational model objective function

Zhigang Tu, Coert van Gemeren, Wei Xie, Remco C. Veltkamp, Jun Cao, Ronald Poppe,.Variational method for joint optical flow estimation and edge-aware image restoration. 65 (0),.

Zhigang Tu, et al."Variational method for joint optical flow estimation and edge-aware image restoration" 65

Zhigang Tu, Coert van Gemeren, Wei Xie, Remco C. Veltkamp, Jun Cao, Ronald Poppe,"Variational method for joint optical flow estimation and edge-aware image restoration"