In this paper, we provide methods for creatively incorporating information from financial news and Twitter feeds into predicting the prices of a portfolio of stocks, using the framework of the Multivariate Bayesian Structural Time Series (MBSTS) model. MBSTS is a Bayesian machine learning model designed to capture correlations among multiple target time series, while using a number of contemporaneous predictors. As an illustration of the current model, we use data on two leading online commerce companies, namely Amazon and eBay, and run extensive empirical experiments to examine which if any, text mining predictors would add to the predictability of a stock price. Evaluation of competing models such as the autoregressive integrated moving average (ARIMA) model, and the recurrent neural network (RNN) model with long short term memory (LSTM), in terms of their performances with respect to cumulative one-step-ahead forecast errors with and without sentimental predictors, were carried out. Our contributions are threefold: Firstly, our model is the first one that successfully incorporated the online text mining to an advanced multivariate Bayesian machine learning time series model, which opens the door of applying both text mining and machine learning simultaneously in modern quantitative finance research; Secondly, under the presence of both modern and classical predictors in both fundamental and technical sense, the polarity of news still adds on a complementary effect; Thirdly, we discover that all models under investigation with sentimental predictors do outperform models without these sentimental predictors, and the MBSTS model with sentimental predictors outperforms all the other models.