Neurodevelopmental impairments are the most common extracardiac morbidities among patients with complex congenital heart disease (CHD) across the lifespan. Robust clinical research in this area has revealed several cardiac, medical, and social factors that can contribute to neurodevelopmental outcome in the context of CHD. Studies using brain magnetic resonance imaging (MRI) have been instrumental in identifying quantitative and qualitative difference in brain structure and maturation in this patient population. Full-term newborns with complex CHD are known to have abnormal microstructural and metabolic brain development with patterns similar to those seen in premature infants at approximately 34 to 36 weeks' gestation. With the advent of fetal brain MRI, these brain abnormalities are now documented as they begin in utero, as early as the third trimester. Importantly, disturbed brain development in utero is now known to be independently associated with neurodevelopmental outcome in early childhood, making the prenatal period an important timeframe for potential interventions. Advances in fetal brain MRI provide a robust imaging tool to use in future neuroprotective clinical trials. The causes of abnormal fetal brain development are multifactorial and include cardiovascular physiology, genetic abnormalities, placental impairment, and other environmental and social factors. This review provides an overview of current knowledge of brain development in the context of CHD, common prenatal imaging tools to evaluate the developing fetal brain in CHD, and known risk factors contributing to brain immaturity.