一种双路并行的大规模手势识别模型

文章以大规模手势为研究对象, 提出一种基于肌电信号(electromyography, EMG)分支和惯性测量单元(inertial measurement unit, IMU)分支的双路并行手势识别模型。首先, 设计双路并行模型来充分提取数据特征, EMG 分支利用二维卷积神经网络设计双流结构, 分别关注 EMG 信号的空间和通道变化, IMU 分支在卷积长短时记忆(convolutional long short-term memory, ConvLSTM)网络基础上引入时间机制, 将空间信息与时间信息融合; 其次, 对模型预训练并根据预训练模型进行参数微调, 提高模型泛化性; 最后, 在 500 个常用的中国手语手势上进行测试, 结果表明, 该模型平均识别率为 82.1%, 与 SignSpeaker 和 CG-Recognizer 相比分别提高了 21.0% 和 6.8%。

In this paper, a dual-path parallel gesture recognition model based on the electromyography (EMG) branch and the inertial measurement unit (IMU) branch is proposed for large-scale gestures. Firstly, the dual-path parallel model is designed to fully extract the data features. The EMG branch uses a two-dimensional convolutional neural network to design a dual-stream structure to focus on the spatial and channel variations of EMG signals, respectively. The IMU branch introduces a temporal mechanism based on the convolutional long short-term memory (ConvLSTM) network to fuse spatial and temporal information. Secondly, the model is pre-trained and the parameters are fine-tuned according to the pre-trained model to improve the generalization of the model. Finally, the model is tested on 500 commonly used Chinese sign language gestures, and the average recognition rate of the model is 82.1%, which is 21.0% and 6.8% higher than that of SignSpeaker and CG-Recognizer, respectively.