Electron flux in the Earth's outer radiation belt is highly variable due to a delicate balance between competing acceleration and loss processes. It has been long recognized that Electromagnetic Ion Cyclotron (EMIC) waves may play a crucial role in the loss of radiation belt electrons. Previous theoretical studies proposed that EMIC waves may account for the loss of the relativistic electron population. However, recent observations showed that while EMIC waves are responsible for the significant loss of ultra-relativistic electrons, the relativistic electron population is almost unaffected. In this study, we provide a theoretical explanation for this discrepancy between previous theoretical studies and recent observations. We demonstrate that EMIC waves mainly contribute to the loss of ultra-relativistic electrons. This study significantly improves the current understanding of the electron dynamics in the Earth's radiation belt and also can help us understand the radiation environments of the exoplanets and outer planets.