中文 | English
Hefei University of Technology Logo J. Hefei Univ. Tech. (Nat. Sci.)

Navigation

Research on lateral control of trajectory tracking based on deep reinforcement learning

Download Full Text

基于深度强化学习的轨迹跟踪横向控制研究

Journal Information

J. Hefei Univ. Tech. (Nat. Sci.), Jul.2023, Vol.46 No.7: 865-872

DOI: 10.3969/j.issn.1003-5060.2023.07.001

Authors

ZHANG Bingli, SHE Yafei

(School of Automobile and Traffic Engineering, Hefei University of Technology, Hefei 230009, China)

Abstract and Keywords

Abstract: In order to explore a controller with better performance and practical application value for the trajectory tracking of autonomous driving, this paper applies the deep reinforcement learning algorithm of twin delayed deep deterministic policy gradient (TD3) to the lateral control of trajectory tracking. The controller design is based on the application scenario of lane line keeping. Firstly, the neural network structure and its parameters are designed based on the TD3 algorithm, and the state space and action output are defined according to the behavior of the human driver, so that it has higher training speed and better control effect. Then, a reward function is designed, which takes tracking accuracy and comfort as the optimization direction of controller performance at the same time. Finally, in order to verify the performance of the designed controller, a variety of simulation experiment scenarios were set up in Prescan to conduct simulation experiments according to the ISO 11270:2014(E) standard. In addition, the comparison with the experimental results of the current main trajectory tracking solutions proves that the controller can meet the application requirements and has better control performance in terms of tracking accuracy and comfort, and has high application value.

Keywords: autonomous driving; trajectory tracking; deep reinforcement learning; twin delayed deep deterministic policy gradient (TD3) algorithm; reward function

作者信息

张炳力，佘亚飞

（合肥工业大学汽车与交通工程学院，安徽 合肥 230009）

摘要和关键词

摘要: 针对自动驾驶的轨迹跟踪问题,为实现性能优异且具有实际应用价值的控制器,文章将双延迟深度确定性策略梯度(twin delayed deep deterministic policy gradient, TD3)的深度强化学习算法应用于轨迹跟踪的横向控制。对车道线保持的应用场景进行控制器设计,首先基于TD3算法对神经网络结构及其参数进行设计,并依据人类驾驶员的行为方式定义状态空间和动作输出,使其具有较快的训练速度以及较好的控制执行效果;然后设计一种奖励函数,将跟踪精度和舒适度同时作为控制器性能的优化方向;最后,根据ISO 11270:2014(E)标准在Prescan中搭建多种使用场景进行仿真实验,验证所设计的控制器性能。通过与当前主流轨迹跟踪解决方案实验结果的对比,分别从跟踪精度和舒适度两方面证明了该控制器可以满足使用要求并且控制性能更加优异,具有的较高应用价值。

关键词: 自动驾驶；轨迹跟踪；深度强化学习；双延迟深度确定性策略梯度（TD3）算法；奖励函数

Funding

安徽省科技重大专项计划资助项目（JZ2022AKKZ0111）

Next
Back to Contents

Account

Submission Login Reviewer Login Editor Login