Comprehensively assessing the effect of a treatment usually includes two objectives, estimating the average treatment effect across the whole target population and evaluating variability of the treatment effect across different subpopulations in an effort to provide more precise treatment recommendations. A common way to identify treatment effect heterogeneity is to split the sample into several strata based on one or more baseline covariates which may be relevant to the effect of treatment, and then compare the localized or stratum-specific treatment effects across those strata. Parametric approaches have been proposed to compare average treatment effects across several strata. One approach is testing interactions between treatment indicator and group indicators in linear regressions (Allison, 1977), and another approach is the likelihood ratio test proposed by Gail and Simon(1985). Both of them require parametric assumptions of outcome distributions. When the parametric assumptions fail, the test may be invalid or the power may be negatively impacted. Thus there is a need for non-parametric tests that can better adapt to various outcome distributions. Randomized experiments are considered to be the gold standard for assessing treatment effects, as all baseline covariates are expected to be well balanced in treatment groups after randomization. However, randomized experiments are not always feasible due to various obstacles, e.g., ethical concerns and high expense. Therefore researchers turn to observational studies. Not only can they avoid the obstacles faced by randomized experiments, but because there are often fewer exclusionary characteristics, the results of observational studies may generalize better to the target population. A main challenge of observational studies is controlling confounding variables. There is a considerable literature on causal inference in observational studies that has been developed targeting this challenge. Many of the proposed procedures balance the observed variables and then rely on the unconfoundedness assumption, i.e., all confounding variables are observed. This assumption is not only strong but also unverifiable. The violation of this assumption can invalidate causal conclusions. A variety of approaches to assessing the sensitivity of causal conclusions to violations of the unconfoundedness assumption have been proposed. By assessing the extent of the assumption violation required to change the conclusion and evaluating the possibility of such a violation based on domain knowledge, it is possible to provide more reliable conclusions. In this dissertation, we describe three contributions we have made to the goal of comprehensively evaluating the effectiveness of treatments. The first two contributions are non-parametric U-statistic-based tests examining the variability of treatment effects across different subpopulations. The first procedure can be appropriately applied in cases, like randomized experiments, where all baseline covariates are well balanced within each stratum; the second procedure adjusts unbalanced confounding variables using propensity scores. Compared to their parametric counterparts, likelihood ratio tests, our non-parametric tests are more powerful when the distributions of study outcomes depart substantially from the distributions assumed by likelihood ratio tests. The third contribution is a sensitivity analysis that addresses the concern of possible violation of the unconfoundedness assumption for the adjusted Mann-Whitney test, a non-parametric test that evaluates the existence of treatment effects in observational studies.

The reliability of a measurement system is studied as a precursor to establishing the accuracy of the measurement system. Forensic science disciplines that rely on feature-based comparisons (e.g., handwriting analysis, fingerprint analysis) have been criticized for the absence of studies demonstrating reliability and accuracy. This has led to empirical evaluations through the use of "black-box" studies. Typically, data collected from inter-examiner (reproducibility) studies is analyzed separately from studies of intra-examiner (repeatability) studies. Motivated by these forensic studies, this dissertation develops methods to assess reliability for continuous, binary, and ordinal outcomes in forensics by combining inter-examiner and intra-examiner data for efficient estimation of reliability, while accounting for possible examiner-forensic sample interactions. Furthermore, we propose an exploratory method to cluster raters/ examiners to identify subpopulations that appear to apply similar decision-making approaches. The dissertation also includes the development of a statistical model to address measurement variability in methylomic studies.

Directional data are observations that can be denoted by unit vectors in Euclidean space and thus are usually used to denote directions. The sample space of a directional variable can be the surface of a circle, a sphere, or a hyper-sphere. Traditional statistical methods are not suitable for analyzing directional data due to the non-Euclidean structure of their sample spaces. In forensic science, the analysis of bloodstain patterns typically requires considering the directions of the constituent bloodstains for crime scene reconstruction. One of the major goals of bloodstain pattern analysis is to evaluate different hypotheses regarding the causal mechanism of a bloodstain pattern. Statistical summaries of the directional attributes of bloodstains can be useful indicators of the causal mechanism. In this dissertation, we first develop an image processing algorithm to approximate bloodstains with ellipses. Afterwards, we employ directional statistics on the orientation of the ellipses to extract useful features and incorporate the distributions of the features into a likelihood ratio framework that has been widely used in various forensic fields to evaluate different hypotheses regarding crime scene evidence. This work provides a conceptual demonstration of the likelihood ratio application to bloodstain pattern analysis. Finally, we extend the approach by proposing a nonparametric Bayesian model that can model directional and linear variables jointly. The resulting Dirichlet process semi-projected normal mixture model can evaluate the likelihood of a bloodstain pattern based on the ellipse representation without pre-designed features.

We were motivated by the two major limitations of the current research approaches on the North Atlantic Oscillation (NAO) based on empirical orthogonal functions (EOF) analysis: (i) long-term stationary assumptions; (ii) lack of measures of uncertainty, and proposed and developed a time-varying low-dimensional representation for spatio-temporal data in this thesis.

The low-dimensional representation is based on a structured spatial covariance matrix using a certain number of structured basis functions with certain parametric forms. Initially, we developed the Parametric Basis Function (PBF) spatial covariance model in a stationary scenario and provided the statistical inference in both maximum likelihood and Bayesian analysis frameworks.

We further extended the model by introducing time-varying parameters to develop the time-varying parametric basis function (TV-PBF) model in the state space model framework. The Bayesian approach with MCMC techniques was used to make inference for the TV-PBF model. The model is able to provide smoothly changing patterns of the 1st EOFs NAO over time which can serve as an alternative representation for the spatio-temporal NAO data.