We study a model of party formation in which party discipline and inter-party ideological heterogeneity are endogenously and jointly determined. Discipline benefits party members because it gives risk-averse voters more confidence in the ideological composition of the party, but this discipline is costly to members who win office. Equilibrium is determined by balancing these forces. We show that this model can account for both comparative difference between parliamentary and presidential systems, and changes over time in the U.S. Congress.