Velocity Planning with Multi-Objectives in Displacement-Time Graphs Using Deep Reinforcement Learning

Abstract

This paper presents a novel velocity planning method in displacement-time graphs with multiple constraints and optimization goals using deep reinforcement learning. The method formulates the velocity planning problem as a reinforcement learning task with state representation including time, position, velocity, acceleration, and distances to each obstacle triangle representative. The action space is discretized within allowable accelerations, and the kinematics ensure velocity constraints during state transitions. The advantage of this method lies in its independence from scene-specific tuning, and exhibiting robustness in various complex scenarios. Comparative analysis demonstrates a 100% success rate, along with superior computational efficiency when contrasted with the baseline approach, while also exhibiting better comfort performance. It offers a valuable alternative for velocity planning in robotics and autonomous vehicles, showcasing deep reinforcement learning’s potential in practical robotics applications.

Liyang Wang

Senior Supervisor R&D Engineer

My research interests include robotics, reinforcement learning, deep learning, path planning, motion planning, trajectory optimization, and controller design.