The rapid adoption of cloud storage and computing services led to unprecedented growth of data centers in the world. As bulk energy consumers, large-scale data centers in the U.S. rack up billions in electricity costs annually. Fortunately, the operational flexibility of data centers can be leveraged to provide valuable frequency regulation services in smart grids to mitigate the indeterminacy of the renewable generation resources. Specifically, this paper aims to leverage computational flexibility provided by servers, such as dynamic voltage frequency scaling and dummy loads. This paper develops a comprehensive framework for data center's frequency regulation service provision in both hour-ahead market and real-time operations. A risk constrained hour-ahead bidding strategy along with a real-time data center power consumption control algorithm are developed to minimize electricity bills and the total response time of the requests. The introduction of dummy load, realistic bi-linear server power consumption model, and probabilistic forecast of electricity and frequency regulation service prices enable the data center to accurately follow frequency regulation signals, while reducing the financial risks associated with electricity market participation. The simulation results show that the proposed frequency regulation provision framework results not only in significant cost reduction for data centers, but also limits degradation in quality of service. Meanwhile, the stability and reliability of a power grid will be improved by the frequency regulation service provision.