In this article, we trace the emergence of the false YIMBY/NIMBY dialectic now dominant in San Francisco housing rights discourse, studying its constitution and material effects. Specifically, we investigate how racial capitalism is constitutive of both YIMBYism and NIMBYism, drawing upon Cedric Robinson’s argument that racialization has always been constitutive of capitalism, and racism is requisite for capitalism’s endurance. We make our argument by drawing upon empirical research conducted by the Anti-Eviction Mapping Project (AEMP), a data analysis, oral history, and critical cartography collective of which we are both a part.  We also draw upon collaborative research between AEMP and community-based housing rights nonprofits and local housing justice organizing efforts, as well as literary and cultural analysis. Such a methodological approach facilitates the unearthing of the racial logics undergirding YIMBYism, pointing to the need for alternative analytics to theorize and mobilize against heightened forms of racialized dispossession. We begin by outlining San Francisco’s YIMBY and NIMBY genealogies, and then proceed to unravel the basic statistical logic underpinning YIMBYism. In doing so, we introduce an additional analytic that we argue is requisite for deconstructing YIMBY algorithms: aesthetic desires of wealthy newcomers. We suggest that the YIMBY “build, baby, build” housing solution fails when architectural and neighborhood fantasies are taken into account. We then study how racialized surveillance informs not only the NIMBY but also the YIMBY gaze, arguing that both camps are ultimately tethered to racial capitalism’s liberal legacies.