People may engage in a joint activity (JA) to accrue material rewards or to help others. From a third-party perspective, the occurrence of JAs is thus ambiguous about the goals of the participating agents. We argue that the payoff structure of a JA (how costs and rewards are distributed) may help disambiguate these goals. Specifically, we hypothesize that an agent’s participation in a JA should be interpreted as prosocially motivated when its costs cannot be recouped by material rewards (disadvantageous payoff). We tested this hypothesis across three looking-time experiments with 12-month-olds. As predicted, infants expected a disadvantaged agent to behave altruistically towards her JA partner (Exp. 1). However, this expectation might be explained by a sensitivity to changes in overall reward distribution from familiarization to test (Exps. 2 & 3). Our results call for a re-evaluation of the role that payoff information plays in early goal attribution within JA contexts.