What do people learn when they repeatedly try to solve a set ofrelated problems? In a set of three different exploratory phys-ical problem solving experiments, participants consistentlylearn strategies rather than generically better world models.Participants selectively transferred these strategies when thecrucial context and preconditions of the strategy were met,such as needing to “catapult”, “support”, “launch” or “desta-bilize” an object in the scene to accomplish their goals. Weshow that these strategies are parameterized: people can ad-just their strategies to account for new object weights despiteno direct interaction experience with these objects. Taken to-gether, these results suggest that people can make use of lim-ited experience to learn abstract strategies that go beyond sim-ple model-free policies and are instead object-oriented, adapt-able, and can be parameterized by model-based variables suchas weight.