Human induced pluripotent stem cells (hiPSCs) provide a unique opportunity to study the human specific aspects of human disorders such as Alzheimer’s disease (AD). However, in vitro systems are inherently artificial and lack the complexity of the in vivo environment. Here we demonstrate how chimeric disease modeling, where we transplanted mixed hiPSC-derived neurons (MNs), comprised primarily of excitatory neurons, and enriched inhibitory neurons (INs) into the mouse hippocampus, allow us to model apoE4 toxicity in different subtypes of human neurons within the in vivo environment. Seven to eight months after transplantation, hiPSC-derived neurons survive and functionally integrate into the mouse brain. However, unlike apoE3 knock-in (apoE3-KI) and apoE4-KI mouse neurons, hiPSC-derived neurons generate A aggregates, with INs generating significantly less than MNs. ApoE4/4 mouse microglia display significant deficits in their ability to phagocytose A aggregates generated by hiPSC-derived neurons, resulting in an increase in the number of aggregates in the apoE4/4 mouse brain. Interestingly, apoE4/4 hiPSC-derived INs are selectively vulnerable to the toxicity of the apoE4/4 brain and display elevated tau-phosphorylation relative to MN transplants. This novel in vivo chimeric model for apoE4/4 toxicity displays human-specific pathological features and allows us to identify how the cellular source of apoE4 relates to pathological response in different neuronal subtypes in the course of Alzheimer’s disease.