UC Riverside

We present Adaptive Skip, an efficient inference-time acceleration technique for large language models that dynamically skips LLM layers using reinforcement learning. Our method formulates layer skipping as a decision-making process where a lightweight policy network determines which transformer layers to compute or skip based on input complexity. Unlike previous approaches, Adaptive Skip maintains full compatibility with KV caching by using consistent skip patterns across tokens and also does not require any retraining of the base LLM. We train the policy using Group Relative Policy Optimization with a multiplicative reward function that jointly optimizes output quality and computational efficiency. Experiments on summarization tasks demonstrate significant speedups (19.4% on CNN/DailyMail and 76.9% on Reddit TLDR) while preserving output quality within 2% of the full model. Adaptive Skip outperforms existing layer skipping methods by learning more effective layer selection patterns adaptive to the input without modifying the underlying language model. Our approach provides a foundation for generalizing to diverse computation modes beyond binary layer skipping.