Many questions remain about the prevalence and effects of SARS-CoV-2 infection in malaria-endemic African countries like Uganda, particularly in vulnerable groups such as pregnant women. We describe SARS-CoV-2 immunoglobulin (Ig)G and IgM antibody responses and clinical outcomes in mother-infant dyads enrolled in malaria chemoprevention trials in Uganda. From December 2020-February 2022, among 400 unvaccinated pregnant women enrolled at 12-20 weeks gestation and followed through delivery, 128 (32%) were seronegative for anti-SARS-CoV-2 IgG and IgM at enrollment and delivery, 80 (20%) were infected prior to or early in pregnancy, and 192 (48%) were infected or re-infected with SARS-CoV-2 during pregnancy. We observed preferential binding of plasma IgG to Wuhan-Hu-1-like antigens in individuals seroconverting up to early 2021, and to Delta variant antigens in a subset of individuals in mid-2021. Breadth of IgG binding to all variants improved over time, consistent with affinity maturation of the antibody response in the cohort. No women experienced severe respiratory illness during the study. SARS-CoV-2 infection in early pregnancy was associated with lower median length-for-age Z-score at age 3 months compared with no infection or late pregnancy infect (-1.54 versus -0.37 and -0.51, P = 0.009). These findings suggest that pregnant Ugandan women experienced high levels of SARS-CoV-2 infection without severe respiratory illness. Variant-specific serology testing demonstrated evidence of antibody affinity maturation at the population level. Early gestational SARS-CoV-2 infection was associated with transient shorter stature in early infancy. Further research should explore the significance of this finding and define targeted measures to prevent infection in pregnancy.