Our understanding of the long-term evolution of the Earth system is based on the assumption that terrestrial weathering rates should respond to, and hence help regulate, atmospheric CO2 and climate. Increased terrestrial weathering requires increased carbonate accumulation in marine sediments, which in turn is expected to result in a long-term deepening of the carbonate compensation depth (CCD). Here, we critically assess this long-term relationship between climate and carbon cycling. We generate a record of marine deep-sea carbonate abundance from selected late Paleocene through early Eocene time slices to reconstruct the position of the CCD. Although our data set allows for a modest CCD deepening, we find no statistically significant change in the CCD despite >3 °C global warming, highlighting the need for additional deep-sea constraints on carbonate accumulation. Using an Earth system model, we show that the impact of warming and increased weathering on the CCD can be obscured by the opposing influences of ocean circulation patterns and sedimentary respiration of organic matter. From our data synthesis and modeling, we suggest that observations of warming, declining δ13C and a relatively stable CCD can be broadly reproduced by mid-Paleogene increases in volcanic CO2 outgassing and weathering. However, remaining data-model discrepancies hint at missing processes in our model, most likely involving the preservation and burial of organic carbon. Our finding of a decoupling between the CCD and global marine carbonate burial rates means that considerable care is needed in attempting to use the CCD to directly gauge global carbonate burial rates and hence weathering rates.