Journal Home > Volume 25 , Issue 1
Tsinghua Science and Technology 2020, 25(1): 68-80 https://doi.org/10.26599/TST.2019.9010004
Open Access | Issue | Published: 22 July 2019

Spotlight: Hot Target Discovery and Localization with Crowdsourced Photos

Show Author's Information Jiaxi Gu,  Jiliang Wang( ),  Lan Zhang,  Zhiwen Yu,  Xiaozhe Xin,  Yunhao Liu
School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China.
School of Software, Tsinghua University, Beijing 100084, China.
School of Computer Science and Technology, University of Science and Technology of China, Hefei 230026, China.
Beijing Sogou Technology Development Co., Ltd., Beijing 100084, China.
Keywords:
crowdsourcing, localization, multimedia, mobile computing
Cite this article:
Gu J, Wang J, Zhang L, et al. Spotlight: Hot Target Discovery and Localization with Crowdsourced Photos. Tsinghua Science and Technology, 2020, 25(1): 68-80. https://doi.org/10.26599/TST.2019.9010004
Download citation
EndNote(RIS)
BibTeX

555

Views

36

Downloads

Citations

5

Crossref

N/A

WoS

6

Scopus

2

CSCD

Abstract Full text About this article

Camera-equipped mobile devices are encouraging people to take more photos and the development and growth of social networks is making it increasingly popular to share photos online. When objects appear in overlapping Fields Of View (FOV), this means that they are drawing much attention and thus indicates their popularity. Successfully discovering and locating these objects can be very useful for many applications, such as criminal investigations, event summaries, and crowdsourcing-based Geographical Information Systems (GIS). Existing methods require either prior knowledge of the environment or intentional photographing. In this paper, we propose a seamless approach called “Spotlight”, which performs passive localization using crowdsourced photos. Using a graph-based model, we combine object images across multiple camera views. Within each set of combined object images, a photographing map is built on which object localization is performed using plane geometry. We evaluate the system’s localization accuracy using photos taken in various scenarios, with the results showing our approach to be effective for passive object localization and to achieve a high level of accuracy.


menu
Abstract
Full text
Outline
About this article

Spotlight: Hot Target Discovery and Localization with Crowdsourced Photos

Show Author's information Jiaxi Gu, Jiliang Wang( ), Lan Zhang, Zhiwen Yu, Xiaozhe Xin, Yunhao Liu
School of Computer Science, Northwestern Polytechnical University, Xi’an 710072, China.
School of Software, Tsinghua University, Beijing 100084, China.
School of Computer Science and Technology, University of Science and Technology of China, Hefei 230026, China.
Beijing Sogou Technology Development Co., Ltd., Beijing 100084, China.

Abstract

Camera-equipped mobile devices are encouraging people to take more photos and the development and growth of social networks is making it increasingly popular to share photos online. When objects appear in overlapping Fields Of View (FOV), this means that they are drawing much attention and thus indicates their popularity. Successfully discovering and locating these objects can be very useful for many applications, such as criminal investigations, event summaries, and crowdsourcing-based Geographical Information Systems (GIS). Existing methods require either prior knowledge of the environment or intentional photographing. In this paper, we propose a seamless approach called “Spotlight”, which performs passive localization using crowdsourced photos. Using a graph-based model, we combine object images across multiple camera views. Within each set of combined object images, a photographing map is built on which object localization is performed using plane geometry. We evaluate the system’s localization accuracy using photos taken in various scenarios, with the results showing our approach to be effective for passive object localization and to achieve a high level of accuracy.

Keywords: crowdsourcing, localization, multimedia, mobile computing

References(35)

[1]
M. Duggan, Photo and video sharing grow online, https://www.pewinternet.org/2013/10/28/photo-and-video-sharing-grow-online/, 2013.
[2]
M. Youssef, M. Mah, and A. Agrawala, Challenges: Device-free passive localization for wireless environments, in Proceedings of the 13th Annual International Conference on Mobile Computing and Networking, Montréal, Canada, 2007, pp. 222-229.
DOI
[3]
X. Zheng, J. Yang, Y. Chen, and Y. Gan, Adaptive device-free passive localization coping with dynamic target speed, in Proceedings of the 32nd International Conference on Computer Communication, Turin, Italy, 2013, pp. 485-489.
DOI
[4]
H. Aly and M. Youssef, New insights into wifi-based device-free localization, in Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Zurich, Switzerland, 2013, pp. 541-548.
DOI
[5]
J. Wang, D. Fang, X. Chen, Z. Yang, T. Xing, and L. Cai, LCS: Compressive sensing based device-free localization for multiple targets in sensor networks, in Proceedings of the 32nd International Conference on Computer Communication, Turin, Italy, 2013, pp. 145-149.
DOI
[6]
C. Liu, D. Fang, Z. Yang, H. Jiang, X. Chen, W. Wang, T. Xing, and L. Cai, RSS distribution-based passive localization and its application in sensor networks, IEEE Transactions on Wireless Communications, vol. 15, no. 4, pp. 2883-2895, 2016.
DOI Google Scholar
[7]
L. M. Ni, D. Zhang, and M. R. Souryal, RFID-based localization and tracking technologies, IEEE Wireless Communications, vol. 18, no. 2, pp. 45-51, 2011.
DOI Google Scholar
[8]
P. Yang, W. Wu, M. Moniri, and C. C. Chibelushi, Efficient object localization using sparsely distributed passive RFID tags, IEEE Transactions on Industrial Electronics, vol. 60, no. 12, pp. 5914-5924, 2013.
DOI Google Scholar
[9]
J. Han, C. Qian, X. Wang, D. Ma, J. Zhao, W. Xi, Z. Jiang, and Z. Wang, Twins: Device-free object tracking using passive tags, IEEE/ACM Transactions on Networking, vol. 24, no. 3, pp. 1605-1617, 2016.
DOI Google Scholar
[10]
K. Wu, J. Xiao, Y. Yi, M. Gao, and L. M. Ni, Fila: Fine-grained indoor localization, in Proceedings of the 31st International Conference on Computer Communication, Orlando, FL, USA, 2012, pp. 2210-2218.
DOI
[11]
J. Xiao, K. Wu, Y. Yi, L. Wang, and L. M. Ni, Pilot: Passive device-free indoor localization using channel state information, in Proceedings of the 33rd International Conference on Distributed Computing Systems, Philadelphia, PA, USA, 2013, pp. 236-245.
DOI
[12]
H. Xu, Z. Yang, Z. Zhou, L. Shangguan, K. Yi, and Y. Liu, Enhancing wifi-based localization with visual clues, in Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan, 2015, pp. 963-974.
DOI
[13]
X. Xiong, Z. Yang, L. Shangguan, Y. Fei, M. Stojmenovic, and Y. Liu, Smartguide: Towards single-image building localization with smartphone, in Proceedings of the 16th International Symposium on Mobile Ad Hoc Networking and Computing, Hangzhou, China, 2015, pp. 117-126.
DOI
[14]
L. Shangguan, Z. Zhou, Z. Yang, K. Liu, Z. Li, X. Zhao, and Y. Liu, Towards accurate object localization with smartphones, IEEE Transactions on Parallel and Distributed Systems, vol. 25, no. 10, pp. 2731-2742, 2014.
DOI Google Scholar
[15]
S. Ren, K. He, R. B. Girshick, and J. Sun, Faster R-CNN: Towards real-time object detection with region proposal networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 6, pp. 1137-1149, 2017.
DOI Google Scholar
[16]
Z. Wu, N. I. Hristov, T. L. Hedrick, T. H. Kunz, and M. Betke, Tracking a large number of objects from multiple views, in Proceedings of the IEEE 12th International Conference on Computer Vision, Kyoto, Japan, 2009, pp. 1546-1553.
[17]
R. Hamid, R. Kumar, J. Hodgins, and I. Essa, A visualization framework for team sports captured using multiple static cameras, Computer Vision and Image Understanding, vol. 118, pp. 171-183, 2014.
DOI Google Scholar
[18]
E. D. Cheng and M. Piccardi, Matching of objects moving across disjoint cameras, in Proceedings of the International Conference on Image Processing, Atlanta, GA, USA, 2006, pp. 1769-1772.
DOI
[19]
X. Wang, G. Doretto, T. Sebastian, J. Rittscher, and P. H. Tu, Shape and appearance context modeling, in Proceedings of the 11th International Conference on Computer Vision, Janeiro, Brazil, 2007, pp. 1-8.
DOI
[20]
P. R. Östergård, A fast algorithm for the maximum clique problem, Discrete Applied Mathematics, vol. 120, nos. 1–3, pp. 197-207, 2002.
DOI Google Scholar
[21]
C. F. Karney, Algorithms for geodesics, Journal of Geodesy, vol. 87, no. 1, pp. 43-55, 2013.
DOI Google Scholar
[22]
K. Kim and L. S. Davis, Multi-camera tracking and segmentation of occluded people on ground plane using search-guided particle filtering, in Proceedings of the 9th European Conference on Computer Vision, Graz, Austria, 2006, pp. 98-109.
DOI
[23]
X. Dai and S. Payandeh, Tracked object association in multi-camera surveillance network, in Proceedings of the International Conference on Systems, Man, and Cybernetics, Manchester, UK, 2013, pp. 4248-4253.
DOI
[24]
Q. Zhai, S. Ding, X. Li, F. Yang, J. Teng, J. Zhu, D. Xuan, Y. F. Zheng, and W. Zhao, VM-tracking: Visual-motion sensing integration for real-time human tracking, in Proceedings of the 34th International Conference on Computer Communication, Hong Kong, China, 2015, pp. 711-719.
DOI
[25]
S. Piva, A. Calbi, D. Angiati, and C. S. Regazzoni, A multi-feature object association framework for overlapped field of view multi-camera video surveillance systems, in Proceedings of the International Conference on Advanced Video and Signal-based Surveillance, Como, Italy, 2005, pp. 505-510.
[26]
T. Liu, Z. Yuan, J. Sun, J. Wang, N. Zheng, X. Tang, and H.-Y. Shum, Learning to detect a salient object, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 2, pp. 353-367, 2011.
DOI Google Scholar
[27]
X. Shen and Y. Wu, A unified approach to salient object detection via low rank matrix recovery, in Proceedings of the Conference on Computer Vision and Pattern Recognition, Providence, RI, USA, 2012, pp. 853-860.
[28]
M. Rubinstein, A. Joulin, J. Kopf, and C. Liu, Unsupervised joint object discovery and segmentation in internet images, in Proceedings of the Conference on Computer Vision and Pattern Recognition, Portland, OR, USA, 2013, pp. 1939-1946.
DOI
[29]
A. Borji, M.-M. Cheng, H. Jiang, and J. Li, Salient object detection: A benchmark, IEEE Transactions on Image Processing, vol. 24, no. 12, pp. 5706-5722, 2015.
DOI Google Scholar
[30]
P. Peng, L. Shou, K. Chen, G. Chen, and S. Wu, The knowing camera 2: Recognizing and annotating places-of-interest in smartphone photos, in Proceedings of the 37th International Conference on Research and Development in Information Retrieval, Gold Coast, Australia, 2014, pp. 707-716.
DOI
[31]
P. Peng, H. Chen, L. Shou, K. Chen, G. Chen, and C. Xu, DeepCamera: A unified framework for recognizing places-of-interest based on deep convnets, in Proceedings of the 24th International Conference on Information and Knowledge Management, Melbourne, Australia, 2015, pp. 1891-1894.
DOI
[32]
P. Peng, L. Shou, K. Chen, G. Chen, and S. Wu, KISS: Knowing camera prototype system for recognizing and annotating places-of-interest, IEEE Transactions on Knowledge and Data Engineering, vol. 28, no. 4, pp. 994-1006, 2016.
DOI Google Scholar
[33]
M. Seifeldin, A. Saeed, A. E. Kosba, A. El-Keyi, and M. Youssef, Nuzzer: A large-scale device-free passive localization system for wireless environments, IEEE Transactions on Mobile Computing, vol. 12, no. 7, pp. 1321-1334, 2013.
DOI Google Scholar
[34]
S. J. Hwang and K. Grauman, Reading between the lines: Object localization using implicit cues from image tags, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 6, pp. 1145-1158, 2012.
DOI Google Scholar
[35]
M. Salek, Y. Bachrach, and P. Key, Hotspotting—A probabilistic graphical model for image object localization through crowdsourcing, in Proceedings of the 27th AAAI Conference on Artificial Intelligence, Bellevue, WA, USA, 2013. pp. 1156-1162.
Publication history
Copyright
Rights and permissions

Publication history

Received: 07 June 2018
Revised: 07 July 2018
Accepted: 12 January 2019
Published: 22 July 2019
Issue date: February 2020

Copyright

© The author(s) 2020

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return