Traditional Physical Performance Tests (PPT) are subjective, labor-intensive and involve on-body sensors for assessing gait and balance disturbances of an individual. Therefore, we developed an unobtrusive, off-body camera based approach to derive objective metrics for gait and balance monitoring. The outcome of our research demonstrates the capability of a computer vision based SPPB assessment tool that can be used as a promising alternative to conventional SPPB protocol to assess and score physical performance objectively and unobtrusively.