Longitudinal polymorphic control of vehicle based on reinforcement learning with prioritized experience extraction

Journal Information

J. Hefei Univ. Tech. (Nat. Sci.), May.2024, Vol.47 No.5: 577-584

DOI: 10.3969/j.issn.1003-5060.2024.05.001

Authors

HUANG He $ ^{1,2,3} $, FU Mengyuan $ ^{1,2,3} $, WU Runchen $ ^{1,2,3} $, HUANG Zechen $ ^{1,2,3} $, ZENG Qi $ ^{1,2,3} $, SHI Qin $ ^{1,2,3} $

(1. School of Automobile and Traffic Engineering, Hefei University of Technology, Hefei 230009, China; 2. Intelligent Manufacturing Institute, Hefei University of Technology, Hefei 230009, China; 3. Engineering Research Center for Intelligent Transportation and Cooperative Vehicle-Infrastructure of Anhui Province, Hefei 230009, China)

Abstract and Keywords

Abstract: In order to solve the polymorphic control problem of the longitudinal motion of vehicle, a deep Q network(DQN) algorithm based on the rule of prioritized experience extraction(PEE) was put forward. The vehicle longitudinal torque transfer model and reinforcement learning algorithm model were analyzed and established. The transplantation of the algorithm and the formulation of reward function were made taking the comprehensive limitations of the relevant factors such as speed and distance into consideration. Through the simulation and hardware-in-the-loop experiment, the effectiveness of deep reinforcement learning algorithm in the longitudinal polymorphic control of vehicle is verified. In addition, PEE rule was introduced to improve the computational efficiency of conventional DQN algorithm and solve the overfitting problem to some extent. The PEE rule also realizes the smooth following speed of the main vehicle, which, in combination with the relative distance, improves the comfort and safety during driving.

Keywords: deep reinforcement learning; longitudinal control; polymorphic control; prioritized experience extraction(PEE) rule

作者信息

黄鹤 $ ^{1,2,3} $，付梦园 $ ^{1,2,3} $，吴润晨 $ ^{1,2,3} $，黄泽辰 $ ^{1,2,3} $，曾琦 $ ^{1,2,3} $，石琴 $ ^{1,2,3} $

(1. 合肥工业大学汽车与交通工程学院，安徽合肥 230009；2. 合肥工业大学智能制造技术研究院，安徽合肥 230009；3. 安徽省智慧交通车路协同工程研究中心，安徽合肥 230009)

摘要和关键词

摘要: 文章提出一种引入经验优先提取（prioritized experience extraction，PEE）规则的深度 Q 网络（deep Q network，DQN）算法，用于解决汽车纵向行驶时的多态控制问题。首先，建立车辆纵向力矩传递模型和强化学习算法模型，在进行算法移植以及制定奖励函数时综合考虑车速、距离等相关因素的综合限制；然后，通过仿真与硬件在环实验验证强化学习算法在汽车纵向多态控制方面的有效性；最后，引入 PEE 规则提高常规 DQN 算法的计算效率，解决算法区域性过拟合问题。PEE 规则的引入有助于平滑主车的跟随车速，与相对距离相配合提升了行驶时的舒适性与安全性。

关键词: 深度强化学习；纵向控制；多态控制；经验优先提取(PEE)规则

Funding

国家自然科学基金资助项目（71971073）；2023年度长三角科技创新共同体联合攻关资助项目（2023CSJGG1600）；安徽省新能源汽车暨智能网联汽车创新工程资助项目（GXXT-2020-076）和2019年度合工大智能院“科技成果转化及产业化”资助项目（IMICZ2019005）

合肥工业大学学报(自然科学版)

Navigation