Colloidal lead halide perovskite nanorods have recently emerged as promising optoelectronic materials. However, more information about how shape anisotropy impacts their complex dielectric function is required to aid the development of applications that take advantage of the strongly polarized absorption and emission. Here, we have determined the anisotropy of the complex dielectric function of CsPbBr3 nanorods by analyzing the ensemble absorption spectra in conjunction with the ensemble spectral fluorescence anisotropy. This strategy allows us to distinguish the absorption of light parallel and perpendicular to the main axis so that the real and imaginary components of the dielectric function along each direction can be determined by the use of an iterative matrix inversion (IMI) methodology. We find that quantum confinement gives rise to unique axis-dependent electronic features in the dielectric function that increase the overall fluorescence anisotropy in addition to the optical anisotropy that results from particle shape, even in the absence of quantum confinement. Further, the procedure outlined here provides a strategy for obtaining anisotropic complex dielectric functions of colloidal materials of varying composition and aspect ratios using ensemble solution-phase spectroscopy.