Even simple, ambiguous signals can have a rich interpretation when viewed in the context of an interaction in a shared environment. We create a model called Paternalistic Communication by combining an existing model of overloaded language -- Rational Speech Acts (RSA) -- with a full agent model of Theory of Mind (ToM). This integration allows signals to be processed in conjunction with common ground in a principled manner dependent on task-dependent action utilities. This modeling perspective treats communication as a way to coordinate diverging perspectives in a cooperative setting. Under Paternalistic Communication, a speaker decides what to say by predicting their partner’s reaction based on the information in common ground and then evaluates those reactions using their own mind which may contain additional information. We demonstrate the flexibility and performance of Paternalistic Communication in a case study with ambiguous signaling through a set of simulations.