Many engineering, machine learning, and scientific issues require black-box optimization since direct analytical gradients are unavailable and objective evaluations are prohibitively expensive. Although existing evolutionary heuristics, Bayesian Optimization approaches, and Energy-Based Models (EBMs) have improved search efficiency, they frequently fail to retain scalability and stability in high-dimensional, dynamically changing landscapes. In this paper, we offer two novel frameworks, EBMBO and REBMBO, that use Bayesian uncertainty quantification, energy-based latent exploration, and reinforcement learning-driven adaptive decision-making to better navigate complex objective spaces. Our rigorous empirical examination reveals dramatically improved sample efficiency, faster convergence, and robust performance across a variety of optimization settings. This study pushes the frontier of black-box optimization by bringing together the capabilities of probabilistic modeling, structured exploration, and adaptive policy updates.