A system based on a hybrid architecture that fuses a Large Language Model (LLM) with Model Predictive Control (MPC) has been implemented to translate natural language commands into vehicle control. It has been validated and deployed in both simulation and real-world vehicle environments.
In the simulation environment, the system was integrated with the ForzaETH Race Stack via a ROS interface, achieving comprehensive autonomous driving simulation testing capabilities.
For real-world deployment, the system has been successfully deployed and experimentally validated on a small-scale vehicle. The test vehicle is equipped with sensors such as radar and cameras and utilizes a hierarchical control architecture consisting of a high-level and a low-level computer. The high-level computer, serving as the onboard computing unit, is responsible for running core algorithms like perception, decision-making, and planning. The low-level computer then precisely executes the control commands sent by the high-level computer through the vehicle's chassis system.