This research focuses on introducing novel machine learning algorithms for predicting the outcome of patients with brain disorders using MR images. We first introduce the challenges in medical image analysis and overview the magnetic resonance imaging, datasets, and existing tools. A brain lesion segmentation method is then presented. The main novelty of this method is a new feature fusion method that integrates location information and the state-of-the-art patch-based neural networks for lesion segmentation. The proposed feature fusion method improves the segmentation performance of the patch-based neural networks. Thereafter, we focus on predicting the overall survival of brain tumor patients. We introduce a novel feature called the tractographic feature to capture the potentially damaged regions due to the presence of the lesion. The tractographic feature is built from the lesion and average connectome information from a group of normal subjects. It takes into account different functional regions that are affected by the lesion, thus complementing the commonly used lesion volume features. The tractographic feature is tested on the Multi-modal Brain Tumor Segmentation (BraTS) 2018 dataset and achieves a better overall survival prediction performance than other features and the gold standard that uses patient age. The proposed tractographic feature is also used to predict the clinical outcome of stroke patients. On the publicly available stroke benchmark, Ischemic Stroke Lesion Segmentation (ISLES) 2017, our proposed tractographic feature achieves higher accuracy than the state-of-the-art feature descriptors. Finally, we focus on predicting the outcome of mild traumatic brain injury (TBI) patients. The tractographic feature cannot be built in this case since the mTBI patients do not have brain lesions in the traditional MR images or CT scans. Here, we present an unsupervised 3D feature clustering algorithm, consisting of 3D dictionary learning, convolutional network, k-means clustering algorithm with group constraint, to gather mild TBI patients into three groups using their structural and diffusion MR images. The proposed method won the 3rd place in Mild Traumatic Brain Injury Outcome Prediction (mTOP) 2016 challenge.