This thesis delves into the world of Functional Data Analysis (FDA) and its analog of Principal Component Analysis (PCA) termed Functional PCA (FPCA). While a brief primer on traditional PCA sets the stage, the main emphasis is on the richness of FDA—a branch of statistics focusing on data represented as curves or surfaces—and the nuances that distinguish it from conventional data analysis techniques. From this foundation, the thesis elaborates on FPCA and its inherent capability to handle the infinite-dimensional nature of functional data, with methodologies rooted in Hilbert spaces. The core exploration revolves around extensions of standard definitions and theorems in statistics, then ends with an application of FPCA in the realm of forecasting. Empirical demonstrations highlight the potential and advantages of utilizing FPCA for prediction tasks. In synthesizing the areas of traditional statistics and functional analysis, this thesis highlights FPCA in the landscape of data analysis.