Bayesian computation of High-Dimensional problems using Markov Chain Monte Carlo (MCMC) or its variants can be extremely slow or completely prohibitive since these methods perform costly computations at each iteration of the sampling chain. While some non-Bayesian alternatives have been somewhat successful in estimation, they struggle to provide uncertainty quantification. These problems are aggravated if the data size is large. To address these problems, the first chapter proposes a novel dynamic feature partitioned regression (DFP) for efficient online inference for high dimensional linear regressions with large or streaming data. DFP constructs a pseudo posterior density of the parameters at every time point and quickly updates the pseudo posterior when a new block of data (data shard) arrives. DFP updates the pseudo posterior at every time point suitably and partitions the set of parameters to exploit parallelization for efficient posterior computation. The proposed approach is applied to high dimensional linear regression models with Gaussian scale mixture priors and spike and slab priors on large parameter spaces, along with large data, and yields state-of-the-art inferential performance. Over time, the algorithm enjoys theoretical support, as pseudo posterior densities get arbitrarily close to the full posterior as the data size grows, as shown in the appendix.

While the first chapter advances methodology for ordinary high dimensional regression, the second chapter focuses on regressions with multiple objects as predictors. Clinical researchers often collect multiple images from separate modalities (sources) to investigate fundamental questions of human health that are inadequately explained by considering one image source at a time. Viewing the collection of images as multiple objects, the successful integration of multi-object data produces a sum of information greater than the individual parts. This chapter is motivated by a multi-modal imaging application where structural/anatomical information from grey matter (GM) and brain connectivity information in the form of a brain connectome network from functional magnetic resonance imaging (fMRI) are available for multiple subjects. The primary goal in this chapter is to develop a regression model to predict a scalar response from multiple objects and to identify regions significantly related to the response. Existing Bayesian regression literature with multi-object predictors either ignores the topology of some/all of these objects or does not adequately make use of the information shared by multiple object predictors. In contrast, this chapter develops a flexible Bayesian regression framework exploiting network information of the brain connectome while leveraging linkages among connectome network and anatomical information from GM to draw inference on significant ROIs and offer predictive inference on the response. The principled Bayesian framework allows precise characterization of the uncertainty in ascertaining a region as influential for predicting the response and the quantification of predictive uncertainty for the response. The framework is implemented using an efficient Markov Chain Monte Carlo algorithm. Empirical results in simulation studies illustrate substantial inferential and predictive gains of the proposed framework over its popular competitors.

While the first two chapters focus on high-dimensional and object-oriented regressions, the third chapter offers a novel clustering technique for high-dimensional tensors with limited sample size. Clustering of high-dimensional tensors with limited sample size has become prevalent in a variety of application areas. Existing Bayesian model-based clustering of tensors yields less accurate clusters when the tensor dimensions are sufficiently large, the sample size is small, and clusters of tensors mainly reveal differences in their variability. This chapter develops a novel clustering technique for high dimensional tensors with limited sample sizes when the clusters show differences in their covariances rather than their means. The proposed approach constructs several matrices from a tensor to adequately estimate its variability along with different modes and implements a model-based approximate Bayesian clustering algorithm with the matrices, thus constructed with the original tensor data. Although some information in the data is discarded, we gain substantial computational efficiency and accuracy in clustering. The simulation study assesses the proposed approach and its competitors in terms of estimating the number of clusters, identifying the modal cluster membership, and the probability of misclassification in clustering (a measure of uncertainty in clustering).Clustering of tensors obtained from EEG data demonstrates an advantage of the proposed approach vis-a-vis its competitors.

Modeling spatial data with flexible statistical models has become an enormously active area of research over the last decade in many disciplines. This work focuses on scaling MC computations for large-scale Bayesian inference in complexspatial models with adequate point estimation and uncertainty in inference and prediction. We first derive a three-step distributed Bayesian inferential framework for multivariate spatial generalized linear mixed effect models (MVspGLMMs) for big data. The proposed approach delivers fully model-based Bayesian parameter inference based on the construction of the “meta posterior” as the Wasserstein Barycenter of pseudo posterior distributions obtained from the partition of the data into independent subsets.

We introduce Bayesian data sketching for spatially varying coefficient regression models (SVCM) to obviate computational challenges presented by large numbers of spatial locations. To address the challenges of analyzing very large spatialdata, we compress spatially oriented data by a random linear transformation to achieve dimension reduction and conduct inference on the compressed data. We establish posterior contraction rates for estimating the spatially varying coefficients and predicting the outcome at new locations under the randomly compressed data model.

Finally, we present a novel idea that employs data sketching for distributed Bayesian inference. The proposed model takes advantage of parallel computation by performing Bayesian inference built on the aggregation of “sketched subsetposteriors". This approach addresses spatial variable selection in SVCMs with big data without developing fundamentally new models or algorithms or making use of any specialized computational hardware. The models are empirically illustrated by simulation experiments and by conducting a spatial analysis of remote sensed vegetation data.

The recent emergence of complex datasets in various disciplines presents a pressing need to devise regression models that have tensors either as a response or as a covariate, often under assumptions of sparsity in the corresponding tensor-valued coefficients. Models that involve tensors often require special treatment in a modeling setting due to their potentially large structures and general assumptions of sparsity in regard to associations with covariates. Importantly, scenarios with small sample sizes benefit from Bayesian methods that allow for flexible model conditions while also rigorously defining the uncertainty in any conclusions drawn from a model.

We begin with general introductions to Bayesian analysis, neuroimaging, and tensor notations in the first chapter. The goal in these short overviews is not to provide a comprehensive background, but to inform the casual reader about key concepts that will be referenced throughout this dissertation. Afterwards, we proceed through new methods in Bayesian modeling of tensor-valued variables, covering three different analysis scenarios.

The goal in the second chapter is to develop a Bayesian tensor response regression in order to identify contiguous spatial regions that are associated with a given covariate. The method is then applied to detecting neuronal activation in functional magnetic resonance imaging (fMRI) experiments in the presence of tensor-valued brain images and a scalar predictor for a single subject. We propose to regress responses from all cells (called voxels in brain activation studies) together as a tensor response on scalar predictors, accounting for the structural information inherent in the tensor response. To estimate model parameters with proper cell specific shrinkage, we propose a novel \emph{multiway stick breaking shrinkage prior} distribution on tensor structured regression coefficients, enabling identification of cells which are related to the predictors. The major novelty of this chapter lies in the theoretical study of the contraction properties for the proposed shrinkage prior in the tensor response regression when the number of cells grows faster than the sample size. Specifically, estimates of tensor regression coefficients are shown to be asymptotically concentrated around the true sparse tensor in $L_2$-sense under mild assumptions. The method is then applied to a single subject within a balloon-analog risk-taking fMRI experiment to make inferences about parts of the subject's brain that are activated by a stimulus.

In the third chapter, the Bayesian tensor response regression is expanded to compare multiple subjects with multiple tensor responses per subject. This allows for inference on a tensor-valued coefficient, as well as correlations between the different tensor response groups. These two types of inference are referred to in neuroimaging as activation and connectivity, respectively. Brain activation and connectivity analyses in task-based fMRI experiments with multiple subjects are currently at the forefront of data-driven neuroscience. In such experiments, interest often lies in understanding activation of brain voxels due to external stimuli and strong association or connectivity between the measurements on a set of pre-specified groups of brain voxels, also known as regions of interest (ROI). This chapter proposes a joint Bayesian additive mixed modeling framework that simultaneously assesses brain activation and connectivity patterns from multiple subjects. In particular, fMRI measurements from each individual obtained in the form of a multi-dimensional array/tensor across time are regressed on functions of the stimuli. A low-rank parallel factorization (PARAFAC) decomposition on the tensor regression coefficients corresponding to the stimuli to achieve parsimony. The multiway stick-breaking shrinkage priors that were developed in the first chapter are employed to infer activation patterns and associated uncertainties in each cell within the tensor responses. Further, the model introduces region specific random effects which are jointly modeled with a Bayesian Gaussian graphical prior to account for the connectivity among pairs of ROIs. Empirical investigations under various simulation studies demonstrate the effectiveness of the method as a tool to simultaneously assess brain activation and connectivity. The method is then applied to the balloon-analog risk-taking fMRI experiment across multiple subjects in order to make inference about how the brain processes risk.

In the fourth chapter, we propose a method to parsimoniously model a scalar response with a tensor-valued covariate using the Tucker tensor decomposition. This method retains the spatial relationship within a tensor-valued covariate, while reducing the number of parameters varying within the model. Simulated data is analyzed to demonstrate model effectiveness, with comparisons made to both classical and Bayesian methods. The method is then applied to data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) to make inferences about the effects of Alzheimer's disease on the brain and to provide a more quantitative framework on which to make diagnoses.

Finally, we conclude with a brief review of the topics covered, research in progress, and future directions for scholastic pursuit.