Journal Home > Volume 4 , issue 4

Identity-recognition technologies require assistive equipment, whereas they are poor in recognition accuracy and expensive. To overcome this deficiency, this paper proposes several gait feature identification algorithms. First, in combination with the collected gait information of individuals from triaxial accelerometers on smartphones, the collected information is preprocessed, and multimodal fusion is used with the existing standard datasets to yield a multimodal synthetic dataset; then, with the multimodal characteristics of the collected biological gait information, a Convolutional Neural Network based Gait Recognition (CNN-GR) model and the related scheme for the multimodal features are developed; at last, regarding the proposed CNN-GR model and scheme, a unimodal gait feature identity single-gait feature identification algorithm and a multimodal gait feature fusion identity multimodal gait information algorithm are proposed. Experimental results show that the proposed algorithms perform well in recognition accuracy, the confusion matrix, and the kappa statistic, and they have better recognition scores and robustness than the compared algorithms; thus, the proposed algorithm has prominent promise in practice.


menu
Abstract
Full text
Outline
About this article

Multimodal Adaptive Identity-Recognition Algorithm Fused with Gait Perception

Show Author's information Changjie WangZhihua Li( )Benjamin Sarpong
Department of Computer Science and Technology, School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, China

Abstract

Identity-recognition technologies require assistive equipment, whereas they are poor in recognition accuracy and expensive. To overcome this deficiency, this paper proposes several gait feature identification algorithms. First, in combination with the collected gait information of individuals from triaxial accelerometers on smartphones, the collected information is preprocessed, and multimodal fusion is used with the existing standard datasets to yield a multimodal synthetic dataset; then, with the multimodal characteristics of the collected biological gait information, a Convolutional Neural Network based Gait Recognition (CNN-GR) model and the related scheme for the multimodal features are developed; at last, regarding the proposed CNN-GR model and scheme, a unimodal gait feature identity single-gait feature identification algorithm and a multimodal gait feature fusion identity multimodal gait information algorithm are proposed. Experimental results show that the proposed algorithms perform well in recognition accuracy, the confusion matrix, and the kappa statistic, and they have better recognition scores and robustness than the compared algorithms; thus, the proposed algorithm has prominent promise in practice.

Keywords:

gait recognition, person identification, deep learning, multimodal feature fusion
Received: 22 October 2020 Revised: 23 March 2021 Accepted: 26 April 2021 Published: 26 August 2021 Issue date: December 2021
References(27)
[1]
M. Haghighat, M. Abdel-Mottaleb, and W. Alhalabi, Discriminant correlation analysis: Real-time feature level fusion for multimodal biometric recognition, IEEE Transactions on Information Forensics and Security, vol. 11, no. 9, pp. 1984-1996, 2016.
[2]
M. P. Murray, A. B. Drought, and R. C. Kory, Walking patterns of normal men, J. Bone Joint Surg. Am., vol. 46, pp. 335-360, 1964.
[3]
G. Johansson, Visual perception of biological motion and a model for its analysis, Perception & Psychophysics, vol. 14, no. 2, pp. 201-211, 1973.
[4]
Y. LeCun and Y. Bengio, Convolutional networks for images, speech, and time-series, in The Handbook of Brain Theory and Neural Networks, M. A. Arbib, ed. Cambridge, MA, UK: The MIT Press, 1995, pp. 276-278.
[5]
S. A. Niyogi and E. H. Adelson, Analyzing and recognizing walking figures in XYT, in Proc. IEEE Conf. Computer Vision & Pattern Recognition, Seattle, WA, USA, 1994, pp. 469-474.
[6]
H. J. Ailisto, M. Lindholm, J. Mantyjarvi, E. Vildjiounaite, and S. M. Makela, Identifying people from gait pattern with accelerometers, in Proc. Volume 5779, Biometric Technology for Human Identification II, Orlando, FL, USA, 2005, pp. 7-14.
[7]
F. Tafazzoli and R. Safabakhsh, Model-based human gait recognition using leg and arm movements, Engineering Applications of Artificial Intelligence, vol. 23, no. 8, pp. 1237-1246, 2010.
[8]
J. N. Mogan, C. P. Lee, and A. W. C. Tan, Gait recognition using temporal gradient patterns, in Proc. 5th Int. Conf. Information and Communication Technology, Melaka, Malaysia, 2017, pp. 1-4.
[9]
E. S. Bigün, J. Biüun, B. Duc, and S. Fischer, Expert conciliation for multi modal person authentication systems by Bayesian statistics, in Proc. of Int. Conf. Audio- and Video-Based Biometric Person Authentication, Crans-Montana, Switzerland, 1997, pp. 291-300.
[10]
A. K. Jain, S. Prabhakar, and S. Y. Chen, Combining multiple matchers for a high security fingerprint verification system, Pattern Recognition Letters, vol. 20, nos. 11-13, pp. 1371-1379, 1999.
[11]
A. K. Jain, S. Prabhakar, and L. Hong, A multichannel approach to fingerprint classification, IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 4, pp. 348-359, 1999.
[12]
A. Baig, A. Bouridane, F. Kurugollu, and G. Qu, Fingerprint-iris fusion based identification system using a single hamming distance matcher, in Proc. of 2009 Symp. Bio-Inspired Learning and Intelligent Systems for Security, Edinburgh, UK, 2009, pp. 9-12.
[13]
Y. Li, D. Ming, L. Wang, H. Z. Qi, and B. K. Wan, Gait and face profile fusion based human identification at a distance in video, (in Chinese), Chinese Journal of Scientific Instrument, vol. 32, no. 2, pp. 264-270, 2011.
[14]
A. I. Bazin and M. S. Nixon, Probabilistic combination of static and dynamic gait features for verification, in Proc. Volume 5779, Biometric Technology for Human Identification II, Orlando, FL, USA, 2005, pp. 23-30.
[15]
D. K. Wagg and M. S. Nixon, Automated markerless extraction of walking people using deformable contour models, Computer Animation and Virtual Worlds, vol. 15, nos. 3&4, pp. 399-406, 2004.
[16]
J. Mantyjarvi, M. Lindholm, E. Vildjiounaite, S. M. Makela, and H. A. Ailisto, Identifying users of portable devices from gait pattern with accelerometers, in Proc. of IEEE Int. Conf. Acoustics, Speech, and Signal Processing, Philadelphia, PA, USA, 2005, pp. ii/973-ii/976.
[17]
L. Bruton and P. Ramamoorthy, Maximally flat low-pass transfer function synthesis using continuous and discrete filters, IEEE Transactions on Circuits & Systems, vol. 23, no. 8, pp. 510-513, 1976.
[18]
N. Stojanović, I. Krstić, N. Stamenković, and G. Perenić, Butterworth transfer function with the equalised group delay response in the maximally flat sense, Electronics Letters, vol. 54, no. 25, pp. 1436-1438, 2018.
[19]
N. Eagle, A. Pentland, and D. Lazer, Inferring friendship network structure by using mobile phone data, Proceedings of the National Academy of Sciences of the United States of America, vol. 106, no. 36, pp. 15274-15278, 2009.
[20]
J. Scott, R. Gass, J. Crowcroft, P. Hui, C. Diot, and A. Chaintreau, CRAWDAD dataset cambridge/haggle (v. 2009-05-29), , 2021.
[21]
J. R. Kwapisz, G. M. Weiss, and S. A. Moore, Activity recognition using cell phone accelerometers, ACM SIGKDD Explorations Newsletter, vol. 12, no. 2, pp. 74-82, 2011.
[22]
J. Cohen, A coefficient of agreement for nominal scales, Educational and Psychological Measurement, vol. 20, no. 1, pp. 37-46, 1960.
[23]
L. L. Liu, Y. L. Yin, W. Qin, and Y. Li, Gait recognition based on outermost contour, International Journal of Computational Intelligence Systems, vol. 4, no. 5, pp. 1090-1099, 2011.
[24]
R. Hecht-Nielsen, Theory of the backpropagation neural network, in Neural Networks for Perception. San Diego, CA, USA: Academic Press, 1992, pp. 65-93.
[25]
Z. F. Wu, Y. Z. Huang, L. Wang, X. G. Wang, and T. N. Tan, A comprehensive study on cross-view gait based human identification with deep CNNs, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 2, pp. 209-226, 2017.
[26]
P. Ebert, An extension of rate-distortion theory to confusion matrices, IEEE Transactions on Information Theory, vol. 14, no. 1, pp. 6-11, 1968.
[27]
D. Simon and D. L. Simon, Analytic confusion matrix bounds for fault detection and isolation using a sum-of-squared-residuals approach, IEEE Transactions on Reliability, vol. 59, no. 2, pp. 287-296, 2010.
Publication history
Copyright
Acknowledgements
Rights and permissions

Publication history

Received: 22 October 2020
Revised: 23 March 2021
Accepted: 26 April 2021
Published: 26 August 2021
Issue date: December 2021

Copyright

© The author(s) 2021

Acknowledgements

This work was supported by the Smart Manufacturing New Model Application Project Ministry of Industry and Information Technology (No. ZH-XZ-18004), Future Research Projects Funds for Science and Technology Department of Jiangsu Province (No. BY2013015-23), the Fundamental Research Funds for the Ministry of Education (No. JUSRP211A 41), the Fundamental Research Funds for the Central Universities (No. JUSRP42003), and the 111 Project (No. B2018).

Rights and permissions

The articles published in this open access journal are distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/).

Reprints and Permission requests may be sought directly from editorial office.

Return