We present a systematic investigation of the magnetoreflectance R(omega,B) of highly oriented pyrolytic graphite in magnetic field B up to 18 T. From these measurements, we report the determination of lifetimes tau associated with the lowest Landau levels in the quantum limit. We find a linear field dependence for inverse lifetime 1/tau(B) of the lowest Landau levels, which is consistent with the hypothesis of a three-dimensional (3D) to 1D crossover in an anisotropic 3D metal in the quantum limit. This enigmatic result uncovers the origin of the anomalous linear in-plane magnetoresistance observed both in bulk graphite and recently in mesoscopic graphite samples.