The diurnal cycle CO$_2$ emissions from fossil fuel combustion and cement production reflect seasonality, weather conditions, working days, and more recently the impact of the COVID-19 pandemic. Here, for the first time we provide a daily CO$_2$ emission dataset for the whole year of 2020 calculated from inventory and near-real-time activity data (called Carbon Monitor project: https://carbonmonitor.org). It was previously suggested from preliminary estimates that did not cover the entire year of 2020 that the pandemics may have caused more than 8% annual decline of global CO$_2$ emissions. Here we show from detailed estimates of the full year data that the global reduction was only 5.4% (-1,901 MtCO$_2$, ). This decrease is 5 times larger than the annual emission drop at the peak of the 2008 Global Financial Crisis. However, global CO$_2$ emissions gradually recovered towards 2019 levels from late April with global partial re-opening. More importantly, global CO$_2$ emissions even increased slightly by +0.9% in December 2020 compared with 2019, indicating the trends of rebound of global emissions. Later waves of COVID-19 infections in late 2020 and corresponding lockdowns have caused further CO$_2$ emissions reductions particularly in western countries, but to a much smaller extent than the declines in the first wave. That even substantial world-wide lockdowns of activity led to a one-time decline in global CO$_2$ emissions of only 5.4% in one year highlights the significant challenges for climate change mitigation that we face in the post-COVID era. These declines are significant, but will be quickly overtaken with new emissions unless the COVID-19 crisis is utilized as a break-point with our fossil-fuel trajectory, notably through policies that make the COVID-19 recovery an opportunity to green national energy and development plans.