As the biggest carbon flux of terrestrial ecosystems from photosynthesis, gross primary productivity (GPP) is an important indicator in understanding the carbon cycle and biogeochemical process of terrestrial ecosystems. Despite advances in remote sensing-based GPP modeling, spatial and temporal variations of GPP are still uncertain especially under extreme climate conditions such as droughts. As the only official products of global spatially explicit GPP, MOD17A2H (GPPMOD) has been widely used to assess the variations of carbon uptake of terrestrial ecosystems. However, systematic assessment of its performance has rarely been conducted especially for the grassland ecosystems where inter-annual variability is high. Based on a collection of GPP datasets (GPPEC) from a global network of eddy covariance towers (FluxNet), we compared GPPMOD and GPPEC at all FluxNet grassland sites with more than five years of observations. We evaluated the performance and robustness of GPPMOD in different grassland biomes (tropical, temperate, and alpine) by using a bootstrapping method for calculating 95% confident intervals (CI) for the linear regression slope, coefficients of determination (R2), and root mean square errors (RMSE). We found that GPPMOD generally underestimated GPP by about 34% across all biomes despite a significant relationship (R2 = 0.66 (CI, 0.63–0.69), RMSE = 2.46 (2.33–2.58) g Cm−2 day−1) for the three grassland biomes. GPPMOD had varied performances with R2 values of 0.72 (0.68–0.75) (temperate), 0.64 (0.59–0.68) (alpine), and 0.40 (0.27–0.52) (tropical). Thus, GPPMOD performed better in low GPP situations (e.g., temperate grassland type), which further indicated that GPPMOD underestimated GPP. The underestimation of GPP could be partly attributed to the biased maximum light use efficiency (εmax) values of different grassland biomes. The uncertainty of the fraction of absorbed photosynthetically active radiation (FPAR) and the water scalar based on the vapor pressure deficit (VPD) could have other reasons for the underestimation. Therefore, more accurate estimates of GPP for different grassland biomes should consider improvements in εmax, FPAR, and the VPD scalar. Our results suggest that the community should be cautious when using MODIS GPP products to examine spatial and temporal variations of carbon fluxes.