In many animal societies, groups of individuals form stable social units that are shaped by well-delineated dominance hierarchies and a range of affiliative relationships. How do socially complex groups maintain cohesion and achieve collective movement? Using high-resolution GPS tracking of members of a wild baboon troop, we test whether collective movement in stable social groups is governed by interactions among local neighbours (commonly found in groups with largely anonymous memberships), social affiliates, and/or by individuals paying attention to global group structure. We construct candidate movement prediction models and evaluate their ability to predict the future trajectory of focal individuals. We find that baboon movements are best predicted by 4 to 6 neighbours. While these are generally individuals' nearest neighbours, we find that baboons have distinct preferences for particular neighbours, and that these social affiliates best predict individual location at longer time scales (>10 minutes). Our results support existing theoretical and empirical studies highlighting the importance of local rules in driving collective outcomes, such as collective departures, in primates. We extend previous studies by elucidating the rules that maintain cohesion in baboons 'on the move', as well as the different temporal scales of social interactions that are at play.