We establish some new relationships between Milnor invariants and Heegaard Floer homology. This includes a formula for the Milnor triple linking number from the link Floer complex, detection results for the Whitehead link and Borromean rings, and a structural property of the $d$-invariants of surgeries on certain algebraically split links.