© 2018 We present an efficient and reliable algorithm for solving a class of nonlinear eigenvalue problems arising from the modeling of particle accelerator cavities. The eigenvalue nonlinearity in these problems results from the use of waveguides to couple external power sources or to allow certain excited electromagnetic modes to exit the cavity. We use a rational approximation to reduce the nonlinear eigenvalue problem first to a rational eigenvalue problem. We then apply a special linearization procedure to turn the rational eigenvalue problem into a larger linear eigenvalue problem with the same eigenvalues, which can be solved by existing iterative methods. By using a compact scheme to represent both the linearized operator and the eigenvectors to be computed, we obtain a numerical method that only involves solving linear systems of equations of the same dimension as the original nonlinear eigenvalue problem. We refer to this method as a compact rational Krylov (CORK) method. We implemented the CORK method in the Omega3P module of the Advanced Computational Electromagnetic 3D Parallel (ACE3P) simulation suite and validated it by comparing the computed cavity resonant frequencies and damping Q factors of a small model problem to those obtained from a fitting procedure that uses frequency responses computed by another ACE3P module called S3P. We also used the CORK method to compute trapped modes damped in an ideal eight 9-cell SRF cavity cryomodule. This was the first time it was possible to compute these modes directly. The damping Q factors of the computed modes match well with those measured in experiments and the difference in resonant frequencies is within the range introduced by cavity imperfection. Therefore, the CORK method is an extremely valuable tool for computational cavity design.