Temporally Coherent Completion of Dynamic Video

Jia-Bin Huang     Sing Bing Kang     Narendra Ahuja     Johannes Kopf    
Virginia Tech     Microsoft Research     University of Illinois, Urbana-Champaign     Facebook

We present an automatic video completion algorithm that synthesizes missing regions in videos in a temporally coherent fashion. Our algorithm can handle dynamic scenes captured using a moving camera. State-of-the-art approaches have difficulties handling such videos because viewpoint changes cause image-space motion vectors in the missing and known regions to be inconsistent. We address this problem by jointly estimating optical flow and color in the missing regions. Using pixel-wise forward/backward flow fields enables us to synthesize temporally coherent colors. We formulate the problem as a non-parametric patch-based optimization. We demonstrate our technique on numerous challenging videos.

[Paper (PDF)] [Supplemental Material] [Masks] [Completion results] [Source code] [BibTex]