Journal Home > Volume 3 , Issue 3
Computational Visual Media 2017, 3(3): 285-294 https://doi.org/10.1007/s41095-017-0083-7
Research Article | Open Access | Issue | Published: 18 May 2017

Robust tracking-by-detection using a selection and completion mechanism

Show Author's Information Ruochen Fan1,  Fang-Lue Zhang2,  Min Zhang3( ),  Ralph R. Martin4
Tsinghua University, Beijing 100084, China.
School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand.
Center of Mathematical Sciences and Applications, Harvard University, Cambridge, Massachusetts, USA.
School of Computer Science and Informatics, Cardiff University, Cardiff, Wales, UK.
Keywords:
detection, object tracking, proposal selection, trajectory completion
Cite this article:
Fan R, Zhang F-L, Zhang M, et al. Robust tracking-by-detection using a selection and completion mechanism. Computational Visual Media, 2017, 3(3): 285-294. https://doi.org/10.1007/s41095-017-0083-7
Download citation
EndNote(RIS)
BibTeX

581

Views

14

Downloads

Citations

7

Crossref

N/A

WoS

9

Scopus

2

CSCD

Abstract Full text About this article

It is challenging to track a target continuously in videos with long-term occlusion, or objects which leave then re-enter a scene. Existing tracking algorithms combined with online-trained object detectors perform unreliably in complex conditions, and can only provide discontinuous trajectories with jumps in position when the object is occluded. This paper proposes a novel framework of tracking-by-detection using selection and completion to solve the abovementioned problems. It has two components, tracking and trajectory completion. An offline-trained object detector can localize objects in the same category as the object being tracked. The object detector is based on a highly accurate deep learning model. The object selector determines which object should be used to re-initialize a traditional tracker. As the object selector is trained online, it allows the framework to be adaptable. During completion, a predictive non-linear autoregressive neural network completes any discontinuous trajectory. The tracking component is an online real-time algorithm, and the completion part is an after-the-event mechanism. Quantitative experiments show a significant improvement in robustness over prior state-of-the-art methods.


menu
Abstract
Full text
Outline
About this article

Robust tracking-by-detection using a selection and completion mechanism

Show Author's information Ruochen Fan1, Fang-Lue Zhang2, Min Zhang3( ), Ralph R. Martin4
Tsinghua University, Beijing 100084, China.
School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand.
Center of Mathematical Sciences and Applications, Harvard University, Cambridge, Massachusetts, USA.
School of Computer Science and Informatics, Cardiff University, Cardiff, Wales, UK.

Abstract

It is challenging to track a target continuously in videos with long-term occlusion, or objects which leave then re-enter a scene. Existing tracking algorithms combined with online-trained object detectors perform unreliably in complex conditions, and can only provide discontinuous trajectories with jumps in position when the object is occluded. This paper proposes a novel framework of tracking-by-detection using selection and completion to solve the abovementioned problems. It has two components, tracking and trajectory completion. An offline-trained object detector can localize objects in the same category as the object being tracked. The object detector is based on a highly accurate deep learning model. The object selector determines which object should be used to re-initialize a traditional tracker. As the object selector is trained online, it allows the framework to be adaptable. During completion, a predictive non-linear autoregressive neural network completes any discontinuous trajectory. The tracking component is an online real-time algorithm, and the completion part is an after-the-event mechanism. Quantitative experiments show a significant improvement in robustness over prior state-of-the-art methods.

Keywords: detection, object tracking, proposal selection, trajectory completion

References(40)

[1]
R. T. Collins,; A. J. Lipton,; H. Fujiyoshi,; T. Kanade, Algorithms for cooperative multisensor surveillance. Proceedings of the IEEE Vol. 89, No. 10, 1456-1477, 2001.
DOI Google Scholar
[2]
M. Greiffenhagen,; D. Comaniciu,; H. Niemann,; V. Ramesh, Design, analysis, and engineering of video monitoring systems: An approach and a case study. Proceedings of the IEEE Vol. 89, No. 10, 1498-1517, 2001.
DOI Google Scholar
[3]
N. K. Kanhere,; S. T. Birchfield,; W. A. Sarasua, Vision based real time traffic monitoring. U.S. Patent 8,379,926. 2013.
[4]
B. T. Morris,; C. Tran,; G. Scora,; M. M. Trivedi,; M. J. Barth, Real-time video-based traffic measurement and visualization system for energy/emissions. IEEE Transactions on Intelligent Transportation Systems Vol. 13, No. 4, 1667-1678, 2012.
DOI Google Scholar
[5]
Y. Rui,; Y. Chen, Better proposal distributions: Object tracking using unscented particle filter. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, II-786-II-793, 2001.
[6]
K. Zhang,; L. Zhang,; M.-H. Yang, Real-time compressive tracking. In: Computer Vision-ECCV 2012. A. Fitzgibbon,; S. Lazebnik,; P. Perona,; Y. Sato,; C. Schmid, Eds. Springer-Verlag Berlin Heidelberg, 864-877, 2012.
[7]
X. Li,; W. Hu,; C. Shen,; Z. Zhang,; A. Dick,; A. van den Hengel, A survey of appearance models in visual object tracking. ACM Transactions on Intelligent Systems and Technology Vol. 4, No. 4, Article No. 58, 2013.
DOI Google Scholar
[8]
M. Isard,; A. Blak, CONDENSATION—Conditional density propagation for visual tracking. International Journal of Computer Vision Vol. 29, No. 1, 5-28, 1998.
DOI Google Scholar
[9]
J. Santner,; C. Leistner,; A. Saffari,; T. Pock,; H. Bischof, PROST: Parallel robust online simple tracking. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 723-730, 2010.
DOI
[10]
M. Hedayati,; M. J. Cree,; J. Scott, Combination of mean shift of colour signature and optical flow for tracking during foreground and background occlusion. In: Image and Video Technology. T. Bräunl,; B. McCane,; M. Rivera,; X. Yu, Eds. Springer International Publishing Switzerland, 87-98, 2016.
DOI
[11]
Q. Zhao,; Z. Yang,; H. Tao, Differential earth mover’s distance with its applications to visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 32, No. 2, 274-287, 2010.
DOI Google Scholar
[12]
C. Sun,; D. Wang,; H. Lu, Occlusion-aware fragment-based tracking with spatial-temporal consistency. IEEE Transactions on Image Processing Vol. 25, No. 8, 3814-3825, 2016.
DOI Google Scholar
[13]
W. Hu,; X. Zhou,; W. Li,; W. Luo,; X. Zhang,; S. Maybank, Active contour-based visual tracking by integrating colors, shapes, and motions. IEEE Transactions on Image Processing Vol. 22, No. 5, 1778-1792, 2013.
DOI Google Scholar
[14]
A. D. Jepson,; D. J. Fleet,; T. F. El-Maraghi, Robust online appearance models for visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 25, No. 10, 1296-1311, 2003.
DOI Google Scholar
[15]
L. Wang,; W. Ouyang,; X. Wang,; H. Lu, STCT: Sequentially training convolutional networks for visual tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1373-1381, 2016.
DOI
[16]
S. Wang,; H. Lu,; F. Yang,; M.-H. Yang, Superpixel tracking. In: Proceedings of the IEEE International Conference on Computer Vision, 1323-1330, 2011.
[17]
D. G. Lowe, Object recognition from local scale-invariant features. In: Proceedings of the 7th IEEE International Conference on Computer Vision, Vol. 2, 1150-1157, 1999.
DOI
[18]
A.-h. Chen,; M. Zhu,; Y.-h. Wang,; C. Xue, Mean shift tracking combining SIFT. In: Proceedings of the 9th International Conference on Signal Processing, 1532-1535, 2008.
[19]
S. Fazli,; H. M. Pour,; H. Bouzari, Particle filter based object tracking with sift and color feature. In: Proceedings of the 2nd International Conference on Machine Vision, 89-93, 2009.
DOI
[20]
H. Zhou,; Y. Yuan,; C. Shi, Object tracking using SIFT features and mean shift. Computer Vision and Image Understanding Vol. 113, No. 3, 345-352, 2009.
DOI Google Scholar
[21]
D. Mahapatra,; M. K. Saini,; Y. Sun, Illumination invariant tracking in office environments using neurobiology-saliency based particle filter. In: Proceedings of the IEEE International Conference on Multimedia and Expo, 953-956, 2008.
DOI
[22]
G. Zhang,; Z. Yuan,; N. Zheng,; X. Sheng,; T. Liu, Visual saliency based object tracking. In: Computer Vision-ACCV 2009. H. Zha,; R. Taniguchi,; S. Maybank, Eds. Springer-Verlag Berlin Heidelberg, 193-203, 2010.
DOI
[23]
Z. W. Kim, Real time object tracking based on dynamic feature grouping with background subtraction. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1-8, 2008.
[24]
O. Williams,; A. Blake,; R. Cipolla, Sparse Bayesian learning for efficient visual tracking. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 27, No. 8, 1292-1304, 2005.
DOI Google Scholar
[25]
Y. Li,; H. Ai,; T. Yamashita,; S. Lao,; M. Kawade, Tracking in low frame rate video: A cascade particle filter with discriminative observers of different life spans. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 30, No. 10, 1728-1740, 2008.
DOI Google Scholar
[26]
K. Okuma,; A. Taleghani,; N. de Freitas,; J. J. Little,; D. G. Lowe, A boosted particle filter: Multitarget detection and tracking. In: Computer Vision-ECCV 2004. T. Pajdla,; J. Matas Eds. Springer-Verlag Berlin Heidelberg, 28-39, 2004.
DOI
[27]
B. Leibe,; K. Schindler,; L. van Gool, Coupled detection and trajectory estimation for multi-object tracking. In: Proceedings of the IEEE 11th International Conference on Computer Vision, 1-8, 2007.
DOI
[28]
H. Grabner,; H. Bischof, On-line boosting and vision. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 260-267, 2006.
[29]
B. Babenko,; M.-H. Yang,; S. Belongie, Visual tracking with online multiple instance learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 983-990, 2009.
DOI
[30]
F. Tang,; S. Brennan,; Q. Zhao,; H. Tao, Co-tracking using semi-supervised support vector machines. In: Proceedings of the IEEE 11th International Conference on Computer Vision, 1-8, 2007.
DOI
[31]
N. Dalal,; B. Triggs, Histograms of oriented gradients for human detection. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, 886-893, 2005.
[32]
R. Girshick,; J. Donahue,; T. Darrell,; J. Malik, Region-based convolutional networks for accurate object detection and segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 1, 142-158, 2016.
DOI Google Scholar
[33]
S. Ren,; K. He,; R. Girshick,; J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks. In: Proceedings of the Advances in Neural Information Processing Systems 28, 91-99, 2015.
[34]
R. Girshick, Fast R-CNN. In: Proceedings of the IEEE International Conference on Computer Vision, 1440-1448, 2015.
DOI
[35]
T. W. S. Chow,; C. T. Leung, Nonlinear autoregressive integrated neural network model for short-term load forecasting. IEE Proceedings-Generation, Transmission and Distribution Vol. 143, No. 5, 500-506, 1996.
DOI Google Scholar
[36]
J. F. Henriques,; R. Caseiro,; P. Martins,; J. Batista, High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 37, No. 3, 583-596, 2015.
DOI Google Scholar
[37]
J. F. Henriques,; R. Caseiro,; P. Martins,; J. Batista, Exploiting the circulant structure of tracking-by-detection with kernels. In: Computer Vision-ECCV 2012. A. Fitzgibbon,; S. Lazebnik,; P. Perona,; Y. Sato,; C. Schmid, Eds. Springer-Verlag Berlin Heidelberg, 702-715, 2012.
DOI
[38]
J. H. Yoon,; M. H. Yang,; K. J. Yoon, Interacting multiview tracker. IEEE Transactions on Pattern Analysis and Machine Intelligence Vol. 38, No. 5, 903-917, 2016.
DOI Google Scholar
[39]
A. Bewley,; Z. Ge,; L. Ott,; F. Ramos,; B. Upcroft, Simple online and realtime tracking. In: Proceedings of the IEEE International Conference on Image Processing, 3464-3468, 2016.
DOI
[40]
C. Ma,; X. Yang,; C. Zhang,; M.-H. Yang, Long-term correlation tracking. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 5388-5396, 2015.
DOI
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Revised: 05 February 2017
Accepted: 07 April 2017
Published: 18 May 2017
Issue date: September 2017

Copyright

© The Author(s) 2017

Acknowledgements

This work was supported by the National Natural Science Foundation of China (Project No. 61521002), the General Financial Grant from the China Postdoctoral Science Foundation (Grant No. 2015M580100), a Research Grant of Beijing Higher Institution Engineering Research Center, and an EPSRC Travel Grant.

Rights and permissions

This article is published with open access at Springerlink.com

The articles published in this journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Other papers from this open access journal are available free of charge from http://www.springer.com/journal/41095. To submit a manuscript, please go to https://www.editorialmanager.com/cvmj.

Return