Clathrin-mediated endocytosis (CME) is a critical trafficking process that begins when an elaborate endocytic protein network is established at the plasma membrane. Interaction of early endocytic proteins with anionic phospholipids and/or cargo has been suggested to trigger CME initiation. However, the exact mechanism by which CME sites are initiated has not been fully elucidated. In the budding yeast Saccharomyces cerevisiae, higher levels of anionic phospholipids and cargo molecules exist in the newly formed daughter cell compared to the levels in the mother cell during polarized growth. Taking advantage of this asymmetry, we quantitatively compared CME proteins in S. cerevisiae mother versus daughter cells, observing differences in the dynamics and composition of key endocytic proteins. Our results show that CME site initiation occurs preferentially on regions of the plasma membrane with a relatively higher density of endocytic cargo and/or acidic phospholipids. Furthermore, our combined live cell-imaging and yeast genetics analysis provided evidence for a molecular mechanism in which CME sites are initiated when Yap1801 and Yap1802 (yeast CALM/AP180) and Syp1 (yeast FCHo1/2) coordinate with anionic phospholipids and cargo molecules to trigger Ede1 (yeast Eps15)-centric CME initiation complex assembly at the plasma membrane.