Optical microresonators with high quality (Q) factors are essential to a wide range of integrated photonic devices. Steady efforts have been directed towards increasing microresonator Q factors across a variety of platforms. With success in reducing microfabrication process-related optical loss as a limitation of Q, the ultimate attainable Q, as determined solely by the constituent microresonator material absorption, has come into focus. Here, we report measurements of the material-limited Q factors in several photonic material platforms. High-Q microresonators are fabricated from thin films of SiO₂, Si₃N₄, Al_0.2Ga_0.8As, and Ta₂O₅. By using cavity-enhanced photothermal spectroscopy, the material-limited Q is determined. The method simultaneously measures the Kerr nonlinearity in each material and reveals how material nonlinearity and ultimate Q vary in a complementary fashion across photonic materials. Besides guiding microresonator design and material development in four material platforms, the results help establish performance limits in future photonic integrated systems.