This paper presents a method to establish a nonlinear temporal correspondence between two video sequences captured by cameras independently moving in a dynamic 3D scene. We assume that the 3D spatial poses of the cameras are known for each frame. With predefined trajectory basis, the coefficients of the reconstructed trajectory of a moving scene point reflect the rhythm in motion. A robust rank constraint from the coefficient matrices is exploited to measure the spatiotemporal alignment quality for every feasible pair of video fragments. Point correspondences across sequences are not required or even it is possible that different points are tracked in different sequences, only if they satisfy the assumption that every 3D point tracked in the observed sequence can be described as a linear combination of a subset of the 3D points tracked in the reference sequence. Synchronization is then performed using a graph-based search algorithm to find the globally optimal path that minimizes both spatial and temporal misalignments. Our algorithm can use both complete and incomplete feature trajectories along time, and is robust to mild outliers. We verify the robustness and performance of the proposed approach on synthetic data as well as on challenging real video sequences.
|Original language||English (US)|
|Number of pages||11|
|Journal||IEEE Transactions on Circuits and Systems for Video Technology|
|State||Published - Nov 2017|
Bibliographical noteFunding Information:
Manuscript received December 25, 2015; revised April 18, 2016; accepted June 11, 2016. Date of publication June 15, 2016; date of current version November 8, 2017. This work was supported by the National Natural Science Foundation of China (NSFC) under Grant 61272287 and Grant 61531014. This paper was recommended by Associate Editor C. Zhang.
- Nonrigid structure from motion
- rank constraint
- trajectory basis
- video synchronization