UCLA

Offline reinforcement learning (RL) aims to learn policies from fixed datasets without further environment interaction. The Decision Transformer (DT) approaches this by modeling trajectories autoregressively, conditioned on return-to-go. The Latent Plan Transformer (LPT) extends this idea by introducing a latent plan variable to guide trajectory generation.This thesis investigates LPT’s ability to achieve fine-grained reward control. We evaluate its accuracy in reaching target rewards, the effect of the classifier guidance parameter ωcg on policy exploitation, and the impact of resampling latent plans during execution.Our results show that while LPT enables flexible control, it underperforms DT on out-of-distribution rewards, likely due to its lack of stepwise supervision. However, resampling improves reward stability, and training on diverse datasets enhances generalization. These insights suggest directions for improving latent-plan-based policies through hybrid modeling and adaptive planning techniques.