Journal Home > Volume 4 , Issue 4
Big Data Mining and Analytics 2021, 4(4): 223-232 https://doi.org/10.26599/BDMA.2021.9020006
Open Access | Issue | Published: 26 August 2021

Multimodal Adaptive Identity-Recognition Algorithm Fused with Gait Perception

Show Author's Information Changjie Wang,  Zhihua Li( ),  Benjamin Sarpong
Department of Computer Science and Technology, School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China
Keywords:
deep learning, gait recognition, person identification, multimodal feature fusion
Cite this article:
Wang C, Li Z, Sarpong B. Multimodal Adaptive Identity-Recognition Algorithm Fused with Gait Perception. Big Data Mining and Analytics, 2021, 4(4): 223-232. https://doi.org/10.26599/BDMA.2021.9020006
Download citation
EndNote(RIS)
BibTeX

842

Views

814

Downloads

Citations

14

Crossref

10

WoS

15

Scopus

1

CSCD

Abstract Full text About this article

Identity-recognition technologies require assistive equipment, whereas they are poor in recognition accuracy and expensive. To overcome this deficiency, this paper proposes several gait feature identification algorithms. First, in combination with the collected gait information of individuals from triaxial accelerometers on smartphones, the collected information is preprocessed, and multimodal fusion is used with the existing standard datasets to yield a multimodal synthetic dataset; then, with the multimodal characteristics of the collected biological gait information, a Convolutional Neural Network based Gait Recognition (CNN-GR) model and the related scheme for the multimodal features are developed; at last, regarding the proposed CNN-GR model and scheme, a unimodal gait feature identity single-gait feature identification algorithm and a multimodal gait feature fusion identity multimodal gait information algorithm are proposed. Experimental results show that the proposed algorithms perform well in recognition accuracy, the confusion matrix, and the kappa statistic, and they have better recognition scores and robustness than the compared algorithms; thus, the proposed algorithm has prominent promise in practice.


menu
Abstract
Full text
Outline
About this article

Multimodal Adaptive Identity-Recognition Algorithm Fused with Gait Perception

Show Author's information Changjie Wang, Zhihua Li( ), Benjamin Sarpong
Department of Computer Science and Technology, School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China

Abstract

Identity-recognition technologies require assistive equipment, whereas they are poor in recognition accuracy and expensive. To overcome this deficiency, this paper proposes several gait feature identification algorithms. First, in combination with the collected gait information of individuals from triaxial accelerometers on smartphones, the collected information is preprocessed, and multimodal fusion is used with the existing standard datasets to yield a multimodal synthetic dataset; then, with the multimodal characteristics of the collected biological gait information, a Convolutional Neural Network based Gait Recognition (CNN-GR) model and the related scheme for the multimodal features are developed; at last, regarding the proposed CNN-GR model and scheme, a unimodal gait feature identity single-gait feature identification algorithm and a multimodal gait feature fusion identity multimodal gait information algorithm are proposed. Experimental results show that the proposed algorithms perform well in recognition accuracy, the confusion matrix, and the kappa statistic, and they have better recognition scores and robustness than the compared algorithms; thus, the proposed algorithm has prominent promise in practice.

Keywords: deep learning, gait recognition, person identification, multimodal feature fusion

References(27)

[1]
M. Haghighat, M. Abdel-Mottaleb, and W. Alhalabi, Discriminant correlation analysis: Real-time feature level fusion for multimodal biometric recognition, IEEE Transactions on Information Forensics and Security, vol. 11, no. 9, pp. 1984-1996, 2016.
DOI Google Scholar
[2]
M. P. Murray, A. B. Drought, and R. C. Kory, Walking patterns of normal men, J. Bone Joint Surg. Am., vol. 46, pp. 335-360, 1964.
DOI Google Scholar
[3]
G. Johansson, Visual perception of biological motion and a model for its analysis, Perception & Psychophysics, vol. 14, no. 2, pp. 201-211, 1973.
DOI Google Scholar
[4]
Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time-series, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, ed. Cambridge, MA, UK: The MIT Press, 1995, pp. 276-278.
[5]
S. A. Niyogi and E. H. Adelson, Analyzing and recognizing walking figures in XYT, in Proc. IEEE Conf. Computer Vision & Pattern Recognition, Seattle, WA, USA, 1994, pp. 469-474.
DOI
[6]
H. J. Ailisto, M. Lindholm, J. Mantyjarvi, E. Vildjiounaite, and S. M. Makela, Identifying people from gait pattern with accelerometers, in Proc. Volume 5779, Biometric Technology for Human Identification II, Orlando, FL, USA, 2005, pp. 7-14.
DOI
[7]
F. Tafazzoli and R. Safabakhsh, Model-based human gait recognition using leg and arm movements, Engineering Applications of Artificial Intelligence, vol. 23, no. 8, pp. 1237-1246, 2010.
DOI Google Scholar
[8]
J. N. Mogan, C. P. Lee, and A. W. C. Tan, Gait recognition using temporal gradient patterns, in Proc. 5th Int. Conf. Information and Communication Technology, Melaka, Malaysia, 2017, pp. 1-4.
DOI
[9]
E. S. Bigün, J. Biüun, B. Duc, and S. Fischer, Expert conciliation for multi modal person authentication systems by Bayesian statistics, in Proc. of Int. Conf. Audio- and Video-Based Biometric Person Authentication, Crans-Montana, Switzerland, 1997, pp. 291-300.
DOI
[10]
A. K. Jain, S. Prabhakar, and S. Y. Chen, Combining multiple matchers for a high security fingerprint verification system, Pattern Recognition Letters, vol. 20, nos. 11-13, pp. 1371-1379, 1999.
DOI Google Scholar
[11]
A. K. Jain, S. Prabhakar, and L. Hong, A multichannel approach to fingerprint classification, IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 4, pp. 348-359, 1999.
DOI Google Scholar
[12]
A. Baig, A. Bouridane, F. Kurugollu, and G. Qu, Fingerprint-iris fusion based identification system using a single hamming distance matcher, in Proc. of 2009 Symp. Bio-Inspired Learning and Intelligent Systems for Security, Edinburgh, UK, 2009, pp. 9-12.
DOI
[13]
Y. Li, D. Ming, L. Wang, H. Z. Qi, and B. K. Wan, Gait and face profile fusion based human identification at a distance in video, (in Chinese), Chinese Journal of Scientific Instrument, vol. 32, no. 2, pp. 264-270, 2011.
Google Scholar
[14]
A. I. Bazin and M. S. Nixon, Probabilistic combination of static and dynamic gait features for verification, in Proc. Volume 5779, Biometric Technology for Human Identification II, Orlando, FL, USA, 2005, pp. 23-30.
DOI
[15]
D. K. Wagg and M. S. Nixon, Automated markerless extraction of walking people using deformable contour models, Computer Animation and Virtual Worlds, vol. 15, nos. 3&4, pp. 399-406, 2004.
DOI Google Scholar
[16]
J. Mantyjarvi, M. Lindholm, E. Vildjiounaite, S. M. Makela, and H. A. Ailisto, Identifying users of portable devices from gait pattern with accelerometers, in Proc. of IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Philadelphia, PA, USA, 2005, pp. ii/973-ii/976.
[17]
L. Bruton and P. Ramamoorthy, Maximally flat low-pass transfer function synthesis using continuous and discrete filters, IEEE Transactions on Circuits & Systems, vol. 23, no. 8, pp. 510-513, 1976.
DOI Google Scholar
[18]
N. Stojanović, I. Krstić, N. Stamenković, and G. Perenić, Butterworth transfer function with the equalised group delay response in the maximally flat sense, Electronics Letters, vol. 54, no. 25, pp. 1436-1438, 2018.
DOI Google Scholar
[19]
N. Eagle, A. Pentland, and D. Lazer, Inferring friendship network structure by using mobile phone data, Proceedings of the National Academy of Sciences of the United States of America, vol. 106, no. 36, pp. 15274-15278, 2009.
DOI Google Scholar
[20]
J. Scott, R. Gass, J. Crowcroft, P. Hui, C. Diot, and A. Chaintreau, CRAWDAD dataset cambridge/haggle (v. 2009-05-29), https://crawdad.org/cambridge/haggle/20090529, 2021.
[21]
J. R. Kwapisz, G. M. Weiss, and S. A. Moore, Activity recognition using cell phone accelerometers, ACM SIGKDD Explorations Newsletter, vol. 12, no. 2, pp. 74-82, 2011.
DOI Google Scholar
[22]
J. Cohen, A coefficient of agreement for nominal scales, Educational and Psychological Measurement, vol. 20, no. 1, pp. 37-46, 1960.
DOI Google Scholar
[23]
L. L. Liu, Y. L. Yin, W. Qin, and Y. Li, Gait recognition based on outermost contour, International Journal of Computational Intelligence Systems, vol. 4, no. 5, pp. 1090-1099, 2011.
DOI Google Scholar
[24]
R. Hecht-Nielsen, Theory of the backpropagation neural network, in Neural Networks for Perception. San Diego, CA, USA: Academic Press, 1992, pp. 65-93.
DOI
[25]
Z. F. Wu, Y. Z. Huang, L. Wang, X. G. Wang, and T. N. Tan, A comprehensive study on cross-view gait based human identification with deep CNNs, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 2, pp. 209-226, 2017.
DOI Google Scholar
[26]
P. Ebert, An extension of rate-distortion theory to confusion matrices, IEEE Transactions on Information Theory, vol. 14, no. 1, pp. 6-11, 1968.
DOI Google Scholar
[27]
D. Simon and D. L. Simon, Analytic confusion matrix bounds for fault detection and isolation using a sum-of-squared-residuals approach, IEEE Transactions on Reliability, vol. 59, no. 2, pp. 287-296, 2010.
DOI Google Scholar
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 22 October 2020
Revised: 23 March 2021
Accepted: 26 April 2021
Published: 26 August 2021
Issue date: December 2021

Copyright

© The author(s) 2021

Acknowledgements

This work was supported by the Smart Manufacturing New Model Application Project Ministry of Industry and Information Technology (No. ZH-XZ-18004), Future Research Projects Funds for Science and Technology Department of Jiangsu Province (No. BY2013015-23), the Fundamental Research Funds for the Ministry of Education (No. JUSRP211A 41), the Fundamental Research Funds for the Central Universities (No. JUSRP42003), and the 111 Project (No. B2018).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Return