Einsteins theory of general relativity predicts that a clock at a higher gravitational potential will tick faster than an otherwise identical clock at a lower potential, an effect known as the gravitational redshift. Here we perform a laboratory-based, blinded test of the gravitational redshift using differential clock comparisons within an evenly spaced array of 5 atomic ensembles spanning a height difference of 1 cm. We measure a fractional frequency gradient of [ - 12.4 ± 0. 7(stat) ± 2. 5(sys)] × 10-19/cm, consistent with the expected redshift gradient of - 10.9 × 10-19/cm. Our results can also be viewed as relativistic gravitational potential difference measurements with sensitivity to mm scale changes in height on the surface of the Earth. These results highlight the potential of local-oscillator-independent differential clock comparisons for emerging applications of optical atomic clocks including geodesy, searches for new physics, gravitational wave detection, and explorations of the interplay between quantum mechanics and gravity.