Sequential data arise in numerous domains, including finance, Natural Language Processing, and healthcare. Effectively modeling and analyzing such data presents significant challenges due to their high dimensionality, nonstationarity, and complex dynamics. This dissertation tackles these challenges by using two powerful mathematical frameworks:signature transformations and neural differential equations (NDEs). The first part explores the versatility of signature transformations in capturing the essential dynamics of sequential data, demonstrating their robustness and effectiveness for tasks such as classification, regression, and time series analysis. The second part investigates the practical applications of NDEs, including neural ordinary differential equations (NODEs) and neural stochastic differential equations (NSDEs), in modeling physical systems, biological processes, and other real-world phenomena with intricate dynamics. The third part delves into the efficacy of Channel Independent (CI) and Channel Dependent (CD) training strategies in multivariate time series forecasting. It presents a rigorous mathematical formulation and theoretical analysis to elucidate why the CI strategy often exceeds the CD strategy, particularly highlighting its robustness to distribution shifts between training and test datasets, a common scenario in real-world applications. Through theoretical analysis, experimental evaluations, and case studies, this dissertation contributes to the advancement of sequential data analysis techniques. It also provides a comprehensive understanding of signature transformations, NDEs, and the impacts of different training strategies, their properties, and their applications in various domains. The findings and methodologies presented in this work have the potential to impact a wide range of fields, allowing more accurate modeling, prediction, and decision-making processes involving sequential data.