To understand the influence of topographical variations in collagen fibril orientation of articular cartilage on optical phase images of polarization-sensitive optical coherence tomography (PS-OCT), we use polarized light microscopy (PLM) to quantify the orientation and phase retardation of the collagen architecture in cartilage at the same locations imaged by PS-OCT. The PS-OCT experiments demonstrate that articular cartilage has normal variations in polarization sensitivity at different locations over an intact bovine tibial plateau. Articular cartilage is not polarization sensitive along the vertical axis on the medial edge and central areas of the joint surface, but becomes polarization sensitive on the lateral edge of the tibia. This difference in optical phase retardation, as demonstrated by PS-OCT, is verified by PLM to be caused by differences in collagen fibril orientation at different locations of the tibial plateau. This study demonstrates that normal topographical variations in the collagen architecture of articular cartilage within a joint have a profound influence on the optical phase retardation detected by PS-OCT imaging, and therefore must be understood and mapped for specific joints before PS-OCT imaging can be used for the evaluation of the health status of individual joint surfaces.