中文 | English

J. Hefei Univ. Tech. (Nat. Sci.)

合肥工业大学学报(自然科学版)

JOURNAL OF HEFEI UNIVERSITY OF TECHNOLOGY (NATURAL SCIENCE)

Competent Authorities: Ministry of Education of the People's Republic of China

Sponsor by: Hefei University of Technology

ISSN 1000-1646, CN 34-1001/N

Long short-term memory-spiking neural network model for sign language recognition

Download Full Text

长短时记忆脉冲神经网络手语识别模型

Journal Information

J. Hefei Univ. Tech. (Nat. Sci.), Nov.2023, Vol.46 No.11: 1479-1483, 1541

DOI: 10.3969/j.issn.1003-5060.2023.11.007

Authors

FENG Yifei, WANG Qingshan

(School of Mathematics, Hefei University of Technology, Hefei 230601, China)

Abstract and Keywords

Abstract: Sign language recognition is an important problem of human-machine interaction. With the development of artificial intelligence, many machine learning and deep learning methods have been applied to sign language recognition tasks. Aiming at the problem of structure complexity of sign language recognition model, a lightweight long short-term memory-spiking neural network (LSTM-SNN) model for sign language recognition was designed. Firstly, adaptive spiking coding was proposed, which converted sign language signals into spiking signals. Then, the spiking signals were input into the improved leaky integrate-and-fire (LIF) model to conduct information transmission in a clock-driven way to complete the training of the network. Experiments on 101 types of sign language datasets were conducted, and the accuracy of the model reached 95.37%, showing that the proposed model is superior to other deep learning and machine learning models.

Keywords: deep learning; pattern recognition; long short-term memory (LSTM); spiking neural network (SNN); sign language recognition

作者信息

冯一飞，王青山

(合肥工业大学数学学院，安徽合肥230601)

摘要和关键词

摘要: 手语识别是人机交互领域中的重要问题之一。随着人工智能技术的发展，越来越多的机器学习和深度学习方法被应用在手语识别任务上。文章设计一种轻量级的长短时记忆脉冲神经网络（long short-term memory-spiking neural network，LSTM-SNN）手语识别模型用于识别常用手语。首先提出自适应脉冲编码，将手语信号转化为脉冲信号；接着将脉冲信号输入到改进的带泄漏整合发放（leaky integrate-and-fire，LIF）神经元模型，以时间驱动的方式进行信息传导，完成网络训练。在收集到的101个手语手势数据集上的实验结果表明，该模型准确率达到95.37%，表明该文提出的模型优于其他深度学习和机器学习模型。

关键词: 深度学习；模式识别；长短时记忆网络（LSTM）；脉冲神经网络（SNN）；手语识别

Funding

中国残联残疾人辅助器具专项研究课题资助项目（CJFJRRB19-2020）

Previous Next

Back to Contents