Using a novel dataset from the TopCoder platform we investigate solvers search for solutions as well as the role of expertisein shaping their problem solving process. We find that while some solvers on the platform do act according to the win-stay, lose-shift rule, skilled solvers are less likely to rely on this meta-heuristic. Somewhat counter-intuitively we find thatexperts make more smaller changes, that is, they change their solutions more often than non-experts, but when they do,they make smaller changes. This can be explained by the fact that experts seem to be able to come up with a good problemrepresentation early on, that doesnt require large adjustments.