Two papers in one episode! Learn about how we can use small amounts of data to train transformers capable of doing truly impressive stuff.

Original Post on X

Dom, cars don’t fly!—Or do they? In-Air Vehicle Maneuver for High-Speed Off-Road Navigation

When pushing the speed limit for aggressive off-road navigation on uneven terrain, it is inevitable that vehicles may become airborne from time to time. During time-sensitive tasks, being able to fly over challenging terrain can also save time, instead of cautiously circumventing or slowly negotiating through. However, most off-road autonomy systems operate under the assumption that the vehicles are always on the ground and therefore limit operational speed. In this paper, we present a novel approach for in-air vehicle maneuver during high-speed off-road navigation. Based on a hybrid forward kinodynamic model using both physics principles and machine learning, our fixed-horizon, sampling-based motion planner ensures accurate vehicle landing poses and their derivatives within a short airborne time window using vehicle throttle and steering commands. We test our approach in extensive in-air experiments both indoors and outdoors, compare it against an error-driven control method, and demonstrate that precise and timely in-air vehicle maneuver is possible through existing ground vehicle controls.

Paper PDF

VERTIFORMER: A Data-Efficient Multi-Task Transformer on Vertically Challenging Terrain

We propose VERTIFORMER, a novel data-efficient multi-task Transformer trained with only one hour of multi-modal data to address the challenges of applying Transformers for robot mobility on extremely rugged, vertically challenging, off-road terrain. With a Transformer encoder and decoder to predict the next robot pose, action, and terrain patch, VERTIFORMER employs a unified state space and missing modality infilling to respectively enhance dynamics understanding and enable a variety of off-road mobility tasks simultaneously, e.g., forward and inverse kinodynamics modeling. By leveraging this unified representation alongside modality infilling, it also achieves real-time task switching during inference for improved fault tolerance and better generalization to unseen environments. Furthermore, VERTIFORMER’s non-autoregressive design also mitigates computational bottlenecks and error propagation associated with autoregressive models. Our experiments offer insights into effectively utilizing Transformers for off-road robot mobility with limited data and demonstrate VERTIFORMER can facilitate multiple off-road mobility tasks onboard a physical mobile robot.

Paper PDF

Open-Source Code

This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit robopapers.substack.com

Ep#14: In-Air Vehicle Maneuver for High-Speed Off-Road Navigation and VERTIFORMER

RoboPapers

Ep#14: In-Air Vehicle Maneuver for High-Speed Off-Road Navigation and VERTIFORMER